Answer-first summary for fast verification
Answer: Implementing stateful aggregations without considering state cleanup mechanisms
Option D is the correct answer because implementing stateful aggregations without considering state cleanup mechanisms can lead to memory leaks and job failure, making it an incorrect approach to optimization. Options A, B, and C are valid strategies for optimizing Spark structured streaming jobs. Increasing the trigger interval (A) can reduce overhead and improve throughput, utilizing watermarks (B) helps manage late data efficiently, and leveraging event hubs' capture feature (C) can preload data for batch processing or backfill, enhancing performance and reliability.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
Which of the following is NOT a recommended strategy for optimizing a Spark structured streaming job that reads from Azure Event Hubs and performs complex event processing?
A
Increasing the trigger interval to reduce processing frequency
B
Utilizing watermarks to handle late-arriving data efficiently
C
Leveraging event hubs' capture feature to preload data
D
Implementing stateful aggregations without considering state cleanup mechanisms
No comments yet.