Explanation:
Option D is the most comprehensive and effective solution. It leverages Databricks' autoscaling feature for dynamic resource adjustment to meet workload demands efficiently, incorporates budget alerts for cost control, and employs a custom monitoring solution for real-time latency tracking. This approach ensures both cost and latency SLAs are met while providing the flexibility to adapt to varying workloads without manual intervention.
Ultimate access to all questions.
In designing a system to ensure cost and latency Service Level Agreements (SLAs) for a production streaming job in Databricks, which of the following approaches provides the BEST balance between cost efficiency and performance optimization? Consider the need for dynamic scalability, real-time monitoring, and the ability to respond to workload fluctuations. Choose one option.
A
Implement auto-scaling to dynamically adjust the number of executors based on the workload, and set up alerts for high latency thresholds without any additional cost monitoring tools.
B
Use Databricks' built-in cost and latency monitoring tools to track performance and costs, requiring manual intervention to adjust the cluster size and configuration as workload changes.
C
Design a system with a fixed number of executors and a static cluster configuration, relying solely on manual monitoring to ensure SLAs are met, ignoring the potential for cost savings through scalability.
D
Utilize Databricks' autoscaling feature to dynamically adjust resources based on workload, implement a budget alert to monitor costs, and deploy a custom monitoring solution to track latency in real-time, triggering alerts when predefined thresholds are exceeded.
No comments yet.