Answer-first summary for fast verification
Answer: Utilize mergeSchema option on each write operation, allowing automatic schema updates.
The correct approach for enforcing schema consistency in a streaming application writing to Delta Lake without disrupting the stream is to utilize the `mergeSchema` option on each write operation. This option automatically updates the table's schema to include new columns from the incoming data, ensuring schema evolution is handled in real-time without manual intervention. This method is efficient, maintains data integrity, and avoids the need for maintenance windows or batch processing for schema validation.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
What is the best approach to enforce schema consistency in a streaming application that writes to Delta Lake, ensuring schema evolution does not disrupt the stream?
A
Stream data to a raw Delta table and use batch jobs for schema validation and data movement to a production table.
B
Implement a schema registry that dynamically updates the Delta table schema before data ingestion.
C
Utilize mergeSchema option on each write operation, allowing automatic schema updates.
D
Disable schema enforcement, relying on manual schema updates during maintenance windows.