Answer-first summary for fast verification
Answer: The application is waiting for resources; increase the number of executors.
The gap between stage completions could be due to the application waiting for resources to become available for the next stage. Increasing the number of executors can help reduce this wait time by providing more resources for task execution.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
You are analyzing the performance of a Spark application using the Spark UI and notice that the event timeline shows a significant gap between the completion of one stage and the start of the next. What could be the cause of this gap, and how would you address it?
A
The application is waiting for resources; increase the number of executors.
B
There is a data dependency issue; optimize the data flow between stages.
C
The application is experiencing a shuffle spill; optimize the shuffle operations.
D
The application is performing complex computations; increase the task parallelism.