Answer-first summary for fast verification
Answer: Deploy the job with comprehensive monitoring and logging, including performance metrics, system health, and error logs, and configure Azure Monitor alerts to notify you of potential node failures in real-time, ensuring quick response to issues.
Option C is the best choice because it provides a balanced approach to ensuring the job's success. Comprehensive monitoring and logging, including performance metrics and system health, allow for early detection of potential issues, such as node failures. Configuring Azure Monitor alerts ensures that you are notified of critical issues in real-time, enabling quick response. This approach not only meets the business SLA by minimizing the risk of job failure but also considers cost efficiency and scalability by leveraging Azure's built-in monitoring tools effectively. The other options either lack sufficient monitoring, risking job failure, or do not fully utilize Azure's capabilities to ensure job reliability and efficiency.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
You are a data engineer responsible for deploying a critical batch job in a production environment on Azure Databricks. The job processes large volumes of data nightly and must complete within a strict 6-hour window to meet business SLAs. Given the importance of the job, you need to ensure its success and minimize the risk of failures, especially those related to node failures. Which of the following strategies would BEST ensure the job's success by providing comprehensive monitoring and proactive failure detection, while also considering cost efficiency and scalability? Choose one option.
A
Deploy the job without any monitoring or logging, relying solely on manual checks the next morning to identify any failures, as this approach minimizes Azure costs.
B
Implement basic logging to capture errors but skip comprehensive monitoring to save on Azure Monitor costs, assuming that node failures are rare and can be addressed manually if they occur.
C
Deploy the job with comprehensive monitoring and logging, including performance metrics, system health, and error logs, and configure Azure Monitor alerts to notify you of potential node failures in real-time, ensuring quick response to issues.
D
Only monitor the job's completion time and ignore other metrics, as this is the only SLA requirement, to reduce complexity and monitoring costs.