Answer-first summary for fast verification
Answer: Checkpointing with RocksDB
Auto Loader stores file metadata in a RocksDB key-value store at the checkpoint location to ensure exactly-once processing.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
Your Auto Loader job is processing files from Azure Data Lake Storage. After a cluster restart, you notice that no files are reprocessed, even though the job was interrupted. Which Auto Loader feature ensures this behavior?
A
File notification mode
B
Directory listing mode
C
Checkpointing with RocksDB
D
Schema evolution