
Ultimate access to all questions.
A data engineering team has an ETL job that runs every midnight but fails intermittently due to a specific task, requiring manual reruns in the morning. This issue is causing significant overhead. What approach can the team take to ensure the job completes every night while minimizing compute costs?
A
Monitor the task during execution to identify the cause of failure
B
Implement a retry policy specifically for the task that fails periodically
C
Apply a retry policy to the entire job
D
Schedule the job to run multiple times to guarantee at least one completion
E
Use a Jobs cluster for each task within the job