The default value of maxFilesPerTrigger is 1000, which can be increased to a much higher number but will require more compute resources to process. Increasing this value reduces the overhead of individual file discovery and setup, allowing Auto Loader to leverage your scaled-up Databricks cluster more effectively for higher throughput. It's essential to find the right balance; setting it too high might strain resources. Other options like merging files, setting up a second Auto Loader process, copying data to local disk, or deeming Auto Loader unsuitable are not the most direct or scalable solutions for this scenario.