Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
A dataset has been defined using Delta Live Tables and includes an expectations clause: CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION FAIL UPDATE
What is the expected behavior when a batch of data containing data that violates these constraints is processed?
Real Exam
Community
KKeng
A
Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
B
Records that violate the expectation cause the job to fail.
C
Records that violate the expectation are dropped from the target dataset and loaded into a quarantine table.
D
Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
E
Records that violate the expectation are added to the target dataset and flagged as invalid in a field added to the target dataset.
Explanation:
Explanation
In Delta Live Tables (DLT), the ON VIOLATION clause determines how data quality expectations are enforced. The specific expectation in the question is:
CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION FAIL UPDATE
CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION FAIL UPDATE
The key part is ON VIOLATION FAIL UPDATE. This means:
- FAIL: When any record violates the constraint, the entire pipeline update (job) will fail.
- UPDATE: This specifies that the failure applies to the update operation.
Therefore, when a batch contains data that violates this constraint, the entire job fails rather than processing the records in any other way.
Let's examine why the other options are incorrect:
- Option A: Incorrect - This describes behavior similar to
ON VIOLATION DROP ROW, notFAIL UPDATE. - Option C: Incorrect - This describes behavior that would require explicit quarantine table configuration, not
FAIL UPDATE. - Option D: Incorrect - This describes behavior similar to
ON VIOLATION RETAIN ROW, notFAIL UPDATE. - Option E: Incorrect - This describes behavior that would require additional configuration for flagging invalid records, not
FAIL UPDATE.
In DLT, there are three main violation actions:
- FAIL UPDATE: Pipeline fails when violations occur
- DROP ROW: Violating rows are dropped
- RETAIN ROW: Violating rows are retained but tracked as invalid
The highlighted answer (B) in the original text indicates this is the correct answer, which aligns with the FAIL UPDATE behavior in Delta Live Tables.
Comments
Loading comments...