
Answer-first summary for fast verification
Answer: No; files containing deleted records may still be accessible with time travel until a VACUUM command is used to remove invalidated data files.
The correct answer is B because Delta Lake's DELETE command logically removes records but does not immediately purge the underlying data files. These files remain accessible via time travel until a VACUUM command is executed, which physically removes the data. This means that while the records are marked for deletion and are not visible in normal operations, they can still be accessed through time travel features until the VACUUM operation is performed. Options A, C, D, and E are incorrect for the reasons provided in the initial explanation.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
The data governance team is evaluating code for GDPR compliance regarding record deletion. The following logic is used to propagate delete requests from the user_lookup table to the user_aggregates table:
(spark.read
.format("delta")
.option("readChangeData", True)
.option("startingTimestamp", '2021-08-22 00:00:00')
.option("endingTimestamp", '2021-08-29 00:00:00')
.table("user_lookup")
.createOrReplaceTempView("changes"))
spark.sql("""
DELETE FROM user_aggregates
WHERE user_id IN (
SELECT user_id
FROM changes
WHERE _change_type = 'delete'
)
""")
(spark.read
.format("delta")
.option("readChangeData", True)
.option("startingTimestamp", '2021-08-22 00:00:00')
.option("endingTimestamp", '2021-08-29 00:00:00')
.table("user_lookup")
.createOrReplaceTempView("changes"))
spark.sql("""
DELETE FROM user_aggregates
WHERE user_id IN (
SELECT user_id
FROM changes
WHERE _change_type = 'delete'
)
""")
Assuming user_id is a unique key and all users requesting deletion have been removed from user_lookup, does successfully executing this logic ensure that the records deleted from user_aggregates are no longer accessible? Explain why.
A
No; the Delta Lake DELETE command only provides ACID guarantees when combined with the MERGE INTO command.
B
No; files containing deleted records may still be accessible with time travel until a VACUUM command is used to remove invalidated data files.
C
Yes; the change data feed uses foreign keys to ensure delete consistency throughout the Lakehouse.
D
Yes; Delta Lake ACID guarantees provide assurance that the DELETE command succeeded fully and permanently purged these records.
E
No; the change data feed only tracks inserts and updates, not deleted records.