
Ultimate access to all questions.
You are tasked with optimizing the performance of a Spark batch job that processes large datasets. Using the Spark UI, you notice that the event timeline shows a high number of small tasks in a specific stage. What steps would you take to optimize this stage?
A
Increase the number of executors to distribute the load.
B
Combine small tasks into larger tasks to reduce task scheduling overhead.
C
Decrease the memory allocation for executors.
D
Increase the shuffle partition size to reduce the number of tasks.