To enhance the efficiency of a batch processing pipeline by analyzing historical execution logs to identify patterns causing longer execution times, which technology stack or approach would be most effective?

Real Exam

Streaming execution logs to Azure Event Hubs and processing them in real-time with Azure Stream Analytics

16.7%

Utilizing Azure Monitor's log analytics workspace to query and visualize log data for pattern identification

32.7%

Exporting logs to Azure Blob Storage and analyzing them using Azure Databricks' Spark SQL for insights

34.7%

Configuring continuous export of logs to Azure Data Lake Storage and using Azure Synapse Analytics for deep log analysis

16.0%

Databricks Certified Data Engineer - Professional