Answer-first summary for fast verification
Answer: spark.sql.files.maxPartitionBytes
The correct parameter is A (spark.sql.files.maxPartitionBytes). This configuration parameter directly affects the size of a Spark partition upon ingestion of data into Spark by setting the maximum number of bytes to pack into a single partition when reading files. The other options either affect different aspects of Spark's behavior or are related to adaptive query execution, which does not directly influence the initial partition size upon data ingestion.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
Which configuration parameter directly controls the size of a Spark partition when data is ingested into Spark?
A
spark.sql.files.maxPartitionBytes
B
spark.sql.autoBroadcastJoinThreshold
C
spark.sql.files.openCostInBytes
D
spark.sql.adaptive.coalescePartitions.minPartitionNum
E
spark.sql.adaptive.advisoryPartitionSizeInBytes