Answer-first summary for fast verification
Answer: To process only new or modified data, thereby enhancing resource efficiency and improving pipeline performance.
The primary goal of incremental processing in the Bronze-to-Silver transition is to handle only the data that has arrived or changed since the last processing interval. By avoiding the re-processing of static historical data, organizations significantly reduce computational overhead, minimize costs, and improve data latency.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
In a Medallion architecture, what is the primary objective of utilizing incremental processing during the transition of data from Bronze to Silver tables?
A
To ensure all historical records are re-processed for auditing purposes.
B
To perform a complete deduplication across the entire table's history with every load.
C
To overwrite the existing Silver table with the full source dataset during each batch run.
D
To process only new or modified data, thereby enhancing resource efficiency and improving pipeline performance.
No comments yet.