In a scenario where the latency of your data pipeline directly affects user experience, which method provides real-time monitoring of this latency to facilitate immediate troubleshooting and optimization?

Real Exam

Streaming job execution logs to Azure Event Hubs and using a real-time analytics service to monitor latency

17.5%

Using the Databricks Spark UI‘s event timeline and job execution details to manually track job latency

11.3%

Configuring Azure Application Insights to track and alert on latency metrics for data processing jobs in Databricks

59.8%

Implementing custom metrics within your data processing jobs in Databricks to track and report latency to Azure Monitor

11.3%

Databricks Certified Data Engineer - Professional