Answer-first summary for fast verification
Answer: Use structured streaming with auto-scaling Databricks clusters based on the streaming query's processing rate.
**Correct Answer: D** - Using structured streaming with auto-scaling Databricks clusters based on the streaming query's processing rate is the optimal strategy for maintaining scalability and performance during peak data influx periods. This approach leverages the power of structured streaming for efficient, fault-tolerant data processing and dynamically adjusts resources to match the workload, ensuring high performance without unnecessary manual intervention or resource wastage.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
You're tasked with designing a real-time data processing solution in Databricks capable of handling streaming data with fluctuating volumes. Which approach guarantees scalability and performance during peak data influx?
A
Preprocess data using Azure Functions before ingesting it into Databricks to reduce load.
B
Archive incoming data during peak periods and process it during off-peak hours.
C
Implement a static number of Spark Streaming receivers regardless of data volume.
D
Use structured streaming with auto-scaling Databricks clusters based on the streaming query's processing rate.