Small files increase the number of input partitions, which inherently improves query performance by maximizing parallelism without any negative impact.

The scanning overhead from reading a large number of small files leads to excessive I/O operations, significantly degrading query performance. The most effective mitigation strategy is to consolidate small files into larger ones or use Delta Lake's optimized file management features.

Over-partitioning the data increases the number of input partitions, which improves query performance by ensuring that each task is lightweight and executes quickly.

Reducing the number of input partitions by under-partitioning the data will decrease parallelism but improve query performance by minimizing I/O operations.