Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
A dataset has been defined using Delta Live Tables and includes an expectations clause: CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION DROP ROW What is the expected behavior when a batch of data containing data that violates these constraints is processed?
Real Exam
Community
KKeng
A
Records that violate the expectation are dropped from the target dataset and loaded into a quarantine table.
B
Records that violate the expectation are added to the target dataset and flagged as invalid in a field added to the target dataset.
C
Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
D
Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
E
Records that violate the expectation cause the job to fail.
Explanation:
Explanation
In Delta Live Tables (DLT), when you define a constraint with ON VIOLATION DROP ROW, the behavior is:
- Records violating the constraint are dropped from the target dataset (not loaded into it)
- Violations are recorded in the event log for monitoring and debugging purposes
- The job continues processing (does not fail)
Let's break down the options:
- Option A: Incorrect - DLT does not automatically load dropped rows into a quarantine table. You would need to explicitly create a separate table for that.
- Option B: Incorrect - This describes
ON VIOLATION FAIL UPDATEbehavior where invalid records are added with a flag. - Option C: CORRECT - This accurately describes
ON VIOLATION DROP ROWbehavior: dropped from target dataset and recorded in event log. - Option D: Incorrect - Records are NOT added to the target dataset with
DROP ROW. - Option E: Incorrect - This would be
ON VIOLATION FAIL UPDATEbehavior, notDROP ROW.
Key DLT Constraint Behaviors:
ON VIOLATION DROP ROW: Drop violating rows, record in event logON VIOLATION FAIL UPDATE: Fail the pipeline if violations exceed threshold- Without
ON VIOLATIONclause: Records are added with metadata about violations
This behavior allows data quality monitoring while ensuring only valid data enters the target dataset.
Comments
Loading comments...