Databricks Certified Data Engineer - Associate

Explanation

The correct answer is C because:

1. ON VIOLATION DROP ROW clause explicitly specifies that rows violating the constraint should be dropped from the target dataset.

2. In Delta Live Tables, when using expectations with ON VIOLATION DROP ROW, the violating records are:

   • Dropped from being written to the target table
   • Recorded in the event log as invalid records
   • Not included in the final dataset

3. Let's analyze why other options are incorrect:

   • A: Incorrect - The job does not fail because ON VIOLATION DROP ROW handles violations gracefully by dropping rows rather than failing the pipeline.
   • B: Incorrect - Records are not added to the target dataset; they are dropped according to the DROP ROW clause.
   • D: Incorrect - Records are not added to the target dataset; they are dropped, though they are indeed recorded in the event log.

4. Key Points:

   • Delta Live Tables provides data quality expectations with different violation policies
   • ON VIOLATION DROP ROW: Drops violating rows and logs them
   • ON VIOLATION FAIL UPDATE: Fails the pipeline when violations exceed a threshold
   • The event log maintains metrics about dropped/invalid records for monitoring and debugging

This behavior ensures data quality while maintaining pipeline resilience by handling invalid data gracefully rather than failing the entire job.