
Answer-first summary for fast verification
Answer: Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
## Explanation The correct answer is **C** because: 1. **ON VIOLATION DROP ROW** clause explicitly specifies that rows violating the constraint should be dropped from the target dataset. 2. In Delta Live Tables, when using expectations with `ON VIOLATION DROP ROW`, the violating records are: - **Dropped** from being written to the target table - **Recorded** in the event log as invalid records - **Not** included in the final dataset 3. Let's analyze why other options are incorrect: - **A**: Incorrect - The job does not fail because `ON VIOLATION DROP ROW` handles violations gracefully by dropping rows rather than failing the pipeline. - **B**: Incorrect - Records are not added to the target dataset; they are dropped according to the `DROP ROW` clause. - **D**: Incorrect - Records are not added to the target dataset; they are dropped, though they are indeed recorded in the event log. 4. **Key Points**: - Delta Live Tables provides data quality expectations with different violation policies - `ON VIOLATION DROP ROW`: Drops violating rows and logs them - `ON VIOLATION FAIL UPDATE`: Fails the pipeline when violations exceed a threshold - The event log maintains metrics about dropped/invalid records for monitoring and debugging This behavior ensures data quality while maintaining pipeline resilience by handling invalid data gracefully rather than failing the entire job.
Author: Keng Suppaseth
Ultimate access to all questions.
A dataset has been defined using Delta Live Tables and includes an expectations clause: CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION DROP ROW
What is the expected behavior when a batch of data containing data that violates these constraints is processed?
A
Records that violate the expectation cause the job to fail.
B
Records that violate the expectation are added to the target dataset and flagged as invalid in a field added to the target dataset.
C
Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
D
Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
No comments yet.