Answer-first summary for fast verification
Answer: Filtering for new or modified records by utilizing metadata such as timestamps or version columns.
Incremental processing focuses on identifying and processing only the records that have changed or been added since the last execution. By leveraging metadata like **timestamps** or **version columns**, data engineers can efficiently filter for new data, avoiding the high cost and overhead of full table scans and full table rewrites.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
Which of the following describes a valid approach for implementing incremental processing during the transition from a Bronze to a Silver layer within a Medallion architecture?
A
Rewriting the entire Silver table using a complete overwrite for every incoming batch.
B
Filtering for new or modified records by utilizing metadata such as timestamps or version columns.
C
Performing a full table scan on the Bronze table to ensure data consistency for every batch.
D
Utilizing watermarks exclusively to process late-arriving data while ignoring standard updates.
No comments yet.