Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
A dataset has been defined using Delta Live Tables and includes an expectations clause:
CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION FAIL UPDATE
CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION FAIL UPDATE
What is the expected behavior when a batch of data containing data that violates these constraints is processed?
Real Exam
Community
KKeng
A
Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
B
Records that violate the expectation cause the job to fail.
C
Records that violate the expectation are dropped from the target dataset and loaded into a quarantine table.
D
Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
E
Records that violate the expectation are added to the target dataset and flagged as invalid in a field added to the target dataset.
Explanation:
The ON VIOLATION FAIL UPDATE clause in Delta Live Tables expectations causes the entire job to fail when any records violate the constraint. This is different from the warn action (which allows invalid records to be written while reporting metrics) or the drop action (which drops invalid records and continues processing). The fail action requires manual intervention before re-processing can occur, making it suitable for critical data quality requirements where invalid data should not be processed at all.
Comments
Loading comments...