Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
A dataset has been defined using Delta Live Tables and includes an expectations clause:
CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION DROP ROW
CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION DROP ROW
What is the expected behavior when a batch of data containing data that violates these constraints is processed?
Real Exam
Community
KKeng
A
Records that violate the expectation are dropped from the target dataset and loaded into a quarantine table.
B
Records that violate the expectation are added to the target dataset and flagged as invalid in a field added to the target dataset.
C
Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
Explanation:
Explanation
In Delta Live Tables (DLT), when you define a constraint with ON VIOLATION DROP ROW, the behavior is:
- Records that violate the constraint are dropped from the target dataset - they are not included in the final output table.
- The violation is recorded in the event log - DLT tracks these violations for monitoring and debugging purposes.
- No quarantine table is automatically created - Option A is incorrect because DLT doesn't automatically load dropped rows into a quarantine table unless explicitly configured to do so.
- Records are not added with flags - Option B is incorrect because the
DROP ROWaction specifically removes the rows, rather than adding them with flags.
Key Points:
ON VIOLATION DROP ROWmeans violating rows are excluded from the target dataset- Violations are logged in the DLT event log for observability
- This is different from
ON VIOLATION FAIL UPDATEwhich would fail the pipeline, or custom handling that might route to quarantine tables - The event log provides visibility into data quality issues without storing the bad data in the main dataset
Comments
Loading comments...