Answer-first summary for fast verification
Answer: Utilize the 'Repair Run' feature in the Jobs UI to execute only the failed tasks and their dependencies.
Databricks provides a built-in 'Repair Run' feature within the Jobs UI specifically designed to optimize recovery. By rerunning only the failed tasks, the system avoids redundant processing of tasks that have already completed successfully, thereby minimizing compute costs and reducing the time to completion.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
A data engineer needs to address failures in a complex Databricks Job while ensuring minimal recomputation and efficient resource usage. Which of the following is the most optimal approach to resolve these failed tasks?
A
Clone the job definition and initiate a fresh rerun of the entire workflow.
B
Restart the compute cluster associated with the job and trigger a full rerun of all tasks.
C
Utilize the 'Repair Run' feature in the Jobs UI to execute only the failed tasks and their dependencies.
D
Manually create a temporary workflow to programmatically handle the logic of the failed tasks.