Answer-first summary for fast verification
Answer: Use the repair feature to rerun only the failed transformation task after resolving the cluster issue.
Utilizing the repair feature allows you to address the failure without having to rerun the successful ingestion part of the job, saving time and preserving the work that has already been completed. This targeted approach is efficient and ensures that the pipeline's integrity is maintained.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
As a data engineer managing a crucial daily data processing pipeline that includes data ingestion, transformation, and loading into a data warehouse, you receive an alert about a job failure during the transformation task due to an unexpected cluster outage. What is the best course of action to minimize downtime and ensure data integrity?
A
Permanently remove the transformation task from the pipeline to avoid future failures.
B
Ignore the alert and wait for the next scheduled run.
C
Manually restart the entire job from the beginning.
D
Use the repair feature to rerun only the failed transformation task after resolving the cluster issue.
No comments yet.