Explanation:
The correct answer is B. While Delta Lake’s time travel feature allows querying of prior table versions, it is intended for short-term retention. The default retention period is 7 days for data files and 30 days for transaction logs.
Using time travel for long-term auditing requires significantly increasing these retention settings, which leads to:
RESTORE become increasingly expensive and slow.In contrast, an SCD Type 2 design allows you to explicitly model history using 'effective' and 'expiry' timestamps. This approach allows for efficient indexing and partitioning, providing performant, cost-controlled access to historical data without the overhead of an unbounded transaction log.
Why the other options are incorrect:
Ultimate access to all questions.
A data architect is considering using Delta Lake’s time travel feature for long-term auditing of valid street addresses in a customers table. They plan to implement a Type 1 table architecture (where records are overwritten) and rely on time travel to retrieve historical states. A data engineer suggests that a Slowly Changing Dimension (SCD) Type 2 table would be a more performant and scalable solution. Which of the following statements provides the critical justification for the data engineer's suggestion?
A
Shallow clones can be utilized in conjunction with Type 1 tables to significantly accelerate historical queries for long-term versioning.
B
Delta Lake time travel is not efficient in terms of cost or latency when used for long-term auditing and data retention.
C
Time travel cannot access previous versions of Type 1 tables because Delta Lake updates modify the underlying data files in place.
D
SCD Type 2 tables require updating multiple fields in a single operation, which increases the risk of data corruption if a query fails during a partial update.
No comments yet.