Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
Deep dive into the quiz with AI chat providers.
We prepare a focused prompt with your quiz and certificate details so each AI can offer a more tailored, in-depth explanation.
A dataset has been defined using Delta Live Tables and includes an expectations clause: CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION FAIL UPDATE
What is the expected behavior when a batch of data containing data that violates these constraints is processed?
Real Exam
Community
KKeng
0
A
Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
B
Records that violate the expectation cause the job to fail.
C
Records that violate the expectation are dropped from the target dataset and loaded into a quarantine table.
D
Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
E
Records that violate the expectation are added to the target dataset and flagged as invalid in a field added to the target dataset.
Explanation:
Explanation
The expectation clause CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION FAIL UPDATE specifies the FAIL action for constraint violations.
In Delta Live Tables, there are three main actions for constraint violations:
- WARN (default) - Invalid records are written to the target dataset, and failures are reported as metrics
- DROP - Invalid records are dropped before writing to the target dataset, and failures are reported as metrics
- FAIL - Invalid records prevent the update from succeeding, requiring manual intervention before re-processing
Since the constraint uses ON VIOLATION FAIL UPDATE, when any record violates the timestamp > '2020-01-01' condition, the entire job will fail. This is the most strict enforcement mode where data quality issues must be resolved before processing can continue.
Key Points:
FAILaction causes the pipeline to stop processing- Manual intervention is required to fix the data quality issue
- This ensures data integrity by preventing invalid data from entering the target dataset
- The job failure will be visible in the Delta Live Tables UI and logs
Comments
Loading comments...