Answer-first summary for fast verification
Answer: Change the checkpoint location to a new path.
The correct solution is to change the checkpoint location because the original checkpoint is tied to the old schema of the upstream table. Changing it allows the query to proceed with the new schema without affecting historical data. Other options either have no effect (since the column is already dropped or the default mode is append) or are incorrect (removing the checkpoint location is not advisable as it's essential for streaming queries).
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
A data engineer executes a query to load data into a downstream table using Delta format. After the upstream table is dropped and recreated without a specific column for compliance, the query fails upon rerun. What modification ensures the query runs successfully while preserving historical data in the downstream table?
A
Change the checkpoint location to a new path.
B
Add .drop('deleted_column_name') before writing to the downstream table.
C
Include .option('mergeSchema', 'true') and remove the checkpoint location option.
D
Add .outputMode('append') to the streaming query.
E
Both add .drop('deleted_column_name') and remove the checkpoint location option.
No comments yet.