Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
A dataset has been defined using Delta Live Tables and includes an expectations clause: CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION DROP ROW What is the expected behavior when a batch of data containing data that violates these constraints is processed?
Real Exam
Community
KKeng
A
Records that violate the expectation are dropped from the target dataset and loaded into a quarantine table.
B
Records that violate the expectation are added to the target dataset and flagged as invalid in a field added to the target dataset.
C
Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
D
Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
E
Records that violate the expectation cause the job to fail.
Explanation:
Explanation
When using Delta Live Tables with the ON VIOLATION DROP ROW clause, the expected behavior is:
- Records violating the constraint are dropped from the target dataset - they are not loaded into the destination table.
- Violations are recorded in the event log - information about dropped records is logged for monitoring and debugging purposes.
- No quarantine table is created - unlike the
ON VIOLATION FAIL UPDATEor other options,DROP ROWsimply removes invalid records without creating separate quarantine tables. - The job continues processing - the pipeline does not fail; it continues processing valid records.
This behavior is documented in Databricks Delta Live Tables expectations documentation. The ON VIOLATION DROP ROW clause specifically means that violating rows are excluded from the target dataset but information about them is recorded in logs for observability.
Why other options are incorrect:
- A: Incorrect because no quarantine table is created with
DROP ROW - B: Incorrect because violating records are not added to the target dataset
- D: Incorrect because violating records are not added to the target dataset
- E: Incorrect because the job does not fail; it continues processing valid records
Comments
Loading comments...