Explanation:
Data Skipping enhances query processing speed in Delta tables. The default setting for delta.dataSkippingNumIndexedCols is 32, limiting statistics collection to the first 32 columns. Adjusting this value upwards increases the number of columns for which statistics are gathered but also raises the overhead during data writes. More details can be found in the documentation on Data Skipping using Z-order.
Ultimate access to all questions.
No comments yet.
To ensure statistics are collected for the first 50 columns of a Delta table with 73 columns, which property needs to be modified to affect future appends?
A
delta.dataSkippingCols
B
The default value is 50; no need to change any property.
C
spark.sql.delta.dataSkippingNumIndexedCols
D
delta.dataSkippingNumIndexedCols
E
spark.sql.dataSkippingNumIndexedCols