You are tasked with designing a production streaming job system on Microsoft Azure that must adhere to strict cost and latency Service Level Agreements (SLAs). The system should efficiently manage resource allocation, ensure fault tolerance, and include robust monitoring mechanisms. Considering the need to balance cost-effectiveness with performance, which of the following strategies would BEST meet these requirements? Choose one option.

Simulated

Allocate the maximum possible resources to the streaming job to minimize latency, without any consideration for cost optimization.

1.7%

Implement a dynamic resource allocation strategy that adjusts based on the current workload, utilizing auto-scaling policies and checkpointing for fault tolerance, alongside comprehensive monitoring to ensure both cost and latency SLAs are met.

95.8%

Use a fixed amount of resources for the streaming job and monitor only latency metrics, ignoring the potential for cost savings through dynamic scaling.

1.3%

Prioritize cost reduction by consistently under-provisioning resources, accepting that this may lead to higher latency and potential SLA violations.

1.3%

Databricks Certified Data Engineer - Professional

Get started today

Comments