
Answer-first summary for fast verification
Answer: Checkpointing and write-ahead logging
Structured Streaming in Apache Spark ensures recovery from failures through two primary techniques: - **Checkpointing**: This saves the progress of the stream at regular intervals. In the event of a failure, Spark can restart from the last checkpoint. - **Write-ahead logging (WAL)**: This records all changes made by a stream processing job before it writes the results to the output. This ensures that if the stream fails, the job can replay the logs to recover lost data. Both of these techniques ensure fault tolerance and data consistency in case of failures. Other options, like 'watermarking,' are used for handling late data and managing event time, but they do not directly relate to failure recovery.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
How does structured streaming ensure recovery from failures during stream processing?
A
Delta time travel
B
Checkpointing and write-ahead logging
C
Write ahead logging and watermarking
D
The stream will failover to available nodes in the cluster
E
Checkpointing and Watermarking
F
Checkpointing and Idempotent sinks
No comments yet.