Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
Which of the following statements best describes the mechanism used by Delta Lake's Optimized Write feature?
A
Before the data is written, a shuffle process is implemented to consolidate similar data, reducing the total number of files generated per partition compared to executors writing files independently.
B
Following the completion of a write, an asynchronous background job examines if files can be further compacted and triggers an OPTIMIZE job, targeting a default size of 1 GB.
C
The OPTIMIZE command is automatically executed on all tables modified during the most recent job session immediately before a Jobs cluster terminates.
D
Optimized writes utilize logical partitions stored in metadata rather than physical directory partitions, effectively eliminating the 'small file' problem by managing boundaries logically.
E
Data is buffered in an external messaging bus instead of being directly committed to memory, with all data being flushed and committed once the job concludes.