Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
A dataset has been defined using Delta Live Tables and includes an expectations clause: CONSTRAINT valid_timestamp EXPECT (timestamp >"2020-01-01") ON VIOLATION FAIL UPDATE What is the expected behavior when a batch of data containing data that violates these constraints is processed?
Real Exam
Community
KKeng
A
The batch will fail, and no data will be written to the dataset.
B
The batch will succeed, and only rows that pass the constraint will be written to the dataset.
C
The batch will succeed, and all rows will be written to the dataset, but rows that violate the constraint will be flagged in a separate log.
D
The batch will succeed, and all rows will be written to the dataset, but rows that violate the constraint will be moved to a quarantine table.
E
The batch will succeed, and all rows will be written to the dataset, but the constraint violation will be recorded in the system metadata.
Explanation:
Explanation
In Delta Live Tables (DLT), constraints with ON VIOLATION FAIL UPDATE specify that the pipeline should fail when data violates the constraint. This is the strictest enforcement mode available in DLT.
Key Points:
ON VIOLATION FAIL UPDATE: This clause indicates that if any rows violate the constraint, the entire update operation (batch) should fail.- Behavior: When a batch contains data that violates this constraint, the pipeline will stop processing that batch, and no data will be written to the target dataset.
- Comparison with other options:
- DROP ROW: Would silently drop violating rows
- FAIL UPDATE: Fails the entire batch when violations are detected
- No data is written when the batch fails
Why other options are incorrect:
- Option B: Incorrect because with
FAIL UPDATE, the batch doesn't succeed when violations exist. - Option C: Incorrect because this describes logging behavior, not failing the batch.
- Option D: Incorrect because quarantine tables are not created with
FAIL UPDATE. - Option E: Incorrect because metadata recording happens with other violation policies, not with
FAIL UPDATE.
Practical Implication:
This enforcement mode ensures data quality by preventing any invalid data from entering the dataset, but requires upstream data validation to prevent pipeline failures.
Comments
Loading comments...