Databricks Certified Data Engineer - Associate

The MERGE command (also known as UPSERT) is used to write data into a Delta table while avoiding duplicate records. It allows you to:

1. Update existing records when a match is found
2. Insert new records when no match is found
3. Delete records when specified conditions are met

This is particularly useful for:

• Incremental data processing where you need to update existing records
• Data deduplication scenarios
• Slowly Changing Dimensions (SCD) implementations

Other options:

• DROP: Deletes the entire table
• INSERT: Adds new records but doesn't prevent duplicates
• APPEND: Adds data to the end of the table without checking for duplicates

The MERGE command uses a source table/view and a target table, and performs conditional updates/inserts based on matching criteria, making it ideal for avoiding duplicate records.