Explanation:
Correct Answer: C
Optimized Writes and Auto Compaction can be enabled globally via Spark configurations. However, Z-order indexing (Z-Ordering) requires table-specific properties to be set and cannot be enabled globally. The correct method is to use:
ALTER TABLE <table_name> SET TBLPROPERTIES ('delta.autoOptimize.zorder.enabled' = 'true');
Incorrect Options:
spark.databricks.delta.autoZorder.enabled and spark.databricks.delta.autoCompact.zorder.enabled) do not exist in Databricks.Ultimate access to all questions.
A data engineer is configuring a pipeline with Optimized Writes and Auto Compaction using the following Spark configurations:
spark.conf.set("spark.databricks.delta.optimizeWrite.enabled", True)
spark.conf.set("spark.databricks.delta.autoCompact.enabled", True)
spark.conf.set("spark.databricks.delta.optimizeWrite.enabled", True)
spark.conf.set("spark.databricks.delta.autoCompact.enabled", True)
They aim to further enhance performance by enabling Z-order indexing with Auto Compaction for data skipping across all pipeline tables. Which solution correctly achieves this?
A
Enable Z-order indexing globally by setting spark.conf.set("spark.databricks.delta.autoZorder.enabled", True).
B
Use spark.conf.set("spark.databricks.delta.autoCompact.zorder.enabled", True) to globally enable Z-order indexing with Auto Compaction.
C
Z-order indexing with Auto Compaction must be enabled individually per table using:
ALTER TABLE table_name SET TBLPROPERTIES (delta.autoOptimize.zorder.enabled = true).
D
No additional configurations are needed; Z-Ordering is automatically enabled with Auto Compaction.
E
Z-order indexing cannot be enabled with Auto Compaction as it does not support Z-Ordering.
No comments yet.