Ultimate access to all questions.
How should a data engineer configure a DLT pipeline with the following streaming tables to retain manually deleted or updated records in the raw_iot
table (which ingests raw device measurements from a heart rate tracker) while ensuring the downstream bpm_stats
table (which computes user statistics from BPM measurements) is recomputed during pipeline updates?
The pipeline consists of:
raw_iot
: Ingests raw heart rate device measurementsbpm_stats
: Incrementally computes user BPM statistics from raw_iot