Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
In a Spark Structured Streaming pipeline using Delta Lake, how is late-arriving data handled when a watermark threshold has been defined?
A
Watermarking ensures that all late-arriving data is eventually processed, prioritizing data completeness over processing latency.
B
Late-arriving data is automatically redirected to a separate side-car table for manual reconciliation and auditing.
C
Data arriving within the defined watermark threshold is processed normally, while data arriving outside the threshold is ignored.
D
Delta Lake ignores the watermark and automatically updates the relevant historical partitions to maintain strict ACID consistency.