Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
In a scenario where a Databricks job fails due to a runtime error in one of its tasks, and you are tasked with repairing and rerunning the job efficiently, considering scalability and automation, which of the following approaches would you choose? The solution should involve identifying the cause of the failure, updating the job configuration, and ensuring minimal manual intervention. Choose the best option from the following:
A
Manually inspect the logs of the failed task through the Databricks UI to identify the error, fix the code, and manually rerun the job. This approach is straightforward but may not be scalable for multiple failures.
B
Utilize the Databricks REST API to programmatically retrieve the job run details, analyze the error logs to identify the cause of the failure, fix the underlying code issue, and then trigger a new job run with the updated configuration automatically. This method supports automation and scalability.
C
Reach out to the Databricks support team for assistance in diagnosing the error and correcting the job configuration. While this ensures expert help, it may introduce delays and lacks automation.
D
Delete the failed job entirely and create a new job from scratch with the corrected code and configuration. This approach avoids dealing with the failure directly but is inefficient and loses historical run data.