Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
A data engineering team is working with a large external Delta table that undergoes frequent changes. They've enabled Optimized Writes and Auto Compaction to automatically compact small data files into target files of 128 MB. Despite this, most data files in the table directory remain smaller than 128 MB. What is the most likely reason for these smaller file sizes?
A
Optimized Writes and Auto Compaction do not affect large Delta tables. Partitioning the table is necessary for Auto Compaction to work at the partition level.
B
Optimized Writes and Auto Compaction are ineffective on external tables. The table must be managed to store file size information in the Hive metastore.
C
Optimized Writes and Auto Compaction create smaller data files to decrease the duration of future MERGE operations.
D
Auto compaction includes Auto Z-Ordering, which is more resource-intensive than simple compaction.
E
The team should check the table’s _auto_optimize directory, where newly compacted files are stored.