Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
In order for Structured Streaming to reliably track the exact progress of the processing so that it can handle any kind of failure by restarting and/or reprocessing, which of the following two approaches is used by Spark to record the offset range of the data being processed in each trigger?
Real Exam
Community
KKeng
A
Checkpointing and Write-ahead Logs
B
Structured Streaming cannot record the offset range of the data being processed in each trigger.
C
Replayable Sources and Idempotent Sinks
D
Write-ahead Logs and Idempotent Sinks
E
Checkpointing and Idempotent Sinks
Explanation:
The engine uses checkpointing and write-ahead logs to record the offset range of the data being processed in each trigger. This approach ensures reliable progress tracking and enables Structured Streaming to handle failures by restarting and/or reprocessing data. Checkpointing saves the state of the streaming query, while write-ahead logs record the offset ranges that have been processed, allowing for exactly-once processing semantics.
Comments
Loading comments...