Answer-first summary for fast verification
Answer: Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
When the Delta Live Tables expectation clause specifies ON VIOLATION DROP ROW, any records that violate the expectation constraints are dropped from the target dataset. These dropped records are then recorded as invalid in the event log. This behavior ensures that only valid records are retained in the target dataset.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
Consider a scenario where you have defined a dataset using Delta Live Tables in Databricks, and this dataset includes an expectations clause to enforce data quality. The specific clause is:
CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION DROP ROW
Given this setup, what would be the expected behavior when a batch of data, which includes rows that do not meet the defined timestamp constraint, is processed?
A
Records that violate the expectation cause the job to fail.
B
Records that violate the expectation are added to the target dataset and flagged as invalid in a field added to the target dataset.
C
Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
D
Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
No comments yet.