Answer-first summary for fast verification
Answer: Set the pipelines.reset.allowed property to false on raw_iot
To retain manually deleted or updated records in the `raw_iot` table, the `pipelines.reset.allowed` property must be set to `false` on `raw_iot` (Option A). This prevents the pipeline from resetting the table during updates, preserving manual changes. Downstream tables like `bpm_stats` will recompute by default during pipeline updates if their own `pipelines.reset.allowed` property is not explicitly disabled. The `skipChangeCommits` flag (Options B/D) is unrelated to retaining manual changes and instead handles CDC scenarios by ignoring update/delete events in the source. Setting `pipelines.reset.allowed` on `bpm_stats` (Option C) is unnecessary, as the goal is to recompute it, which requires allowing resets (the default behavior).
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
How should a data engineer configure a DLT pipeline with the following streaming tables to retain manually deleted or updated records in the raw_iot table (which ingests raw device measurements from a heart rate tracker) while ensuring the downstream bpm_stats table (which computes user statistics from BPM measurements) is recomputed during pipeline updates?
The pipeline consists of:
raw_iot: Ingests raw heart rate device measurementsbpm_stats: Incrementally computes user BPM statistics from raw_iotA
Set the pipelines.reset.allowed property to false on raw_iot
B
Set the skipChangeCommits flag to true on raw_iot
C
Set the pipelines.reset.allowed property to false on bpm_stats
D
Set the skipChangeCommits flag to true on bpm_stats
No comments yet.