Answer-first summary for fast verification
Answer: Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
## Explanation In Delta Live Tables, when using the `ON VIOLATION DROP ROW` clause with a constraint, the behavior is: 1. **Records that violate the constraint are dropped** - They are not included in the target dataset 2. **Violations are recorded in the event log** - Information about dropped records is logged for monitoring and debugging purposes This is different from other options: - **Option A (Job fails)**: This would happen with `ON VIOLATION FAIL UPDATE` or without any `ON VIOLATION` clause (default behavior) - **Option B (Added with flag)**: This would require a different approach like adding a validation column - **Option D (Added to dataset)**: This contradicts the `DROP ROW` directive The `ON VIOLATION DROP ROW` clause provides a way to automatically filter out invalid records while maintaining a record of what was removed, which is useful for data quality management without failing the entire pipeline.
Author: Keng Suppaseth
Ultimate access to all questions.
A dataset has been defined using Delta Live Tables and includes an expectations clause:
CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION DROP ROW
What is the expected behavior when a batch of data containing data that violates these constraints is processed?
A
Records that violate the expectation cause the job to fail.
B
Records that violate the expectation are added to the target dataset and flagged as invalid in a field added to the target dataset.
C
Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
D
Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
No comments yet.