Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
Your Spark application is experiencing performance issues due to large shuffle operations during aggregation. Which of the following strategies would most effectively reduce the size of these shuffle operations?
A
Use coalesce to reduce the number of partitions just before aggregation.
B
Increase the default parallelism to create more tasks and thus reduce data per task.
C
Apply repartition based on the aggregation key before performing the aggregation.
D
Aggregate data locally on each partition with mapPartitions before the global aggregation.