
Ultimate access to all questions.
Which of the following statements best describes the mechanism used by Delta Lake's Optimized Write feature?
A
Before the data is written, a shuffle process is implemented to consolidate similar data, reducing the total number of files generated per partition compared to executors writing files independently.
B
Following the completion of a write, an asynchronous background job examines if files can be further compacted and triggers an OPTIMIZE job, targeting a default size of 1 GB.
C
The OPTIMIZE command is automatically executed on all tables modified during the most recent job session immediately before a Jobs cluster terminates.
D
Optimized writes utilize logical partitions stored in metadata rather than physical directory partitions, effectively eliminating the 'small file' problem by managing boundaries logically.
E
Data is buffered in an external messaging bus instead of being directly committed to memory, with all data being flushed and committed once the job concludes.