Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
In the context of optimizing a data pipeline that processes large volumes of data into a Delta table, consider the following scenario: The pipeline is experiencing performance bottlenecks during both read and write operations. The organization prioritizes cost-efficiency and requires compliance with data governance policies. Given these constraints, which Delta Lake feature should be leveraged to most effectively improve the pipeline's performance while adhering to the organization's priorities? Choose the best option from the following:
A
Increase the size of the transaction log to reduce the number of log entries, thereby minimizing the overhead on write operations.
B
Implement Z-Order clustering on the Delta table to optimize the physical layout of the data, which can significantly reduce the amount of data scanned during read operations and improve write efficiency by co-locating related data.
C
Disable the transaction log to eliminate the overhead associated with logging changes, though this would compromise the ability to audit changes and roll back transactions.
D
Partition the Delta table by a column that is frequently used in query predicates to improve query performance, but this may not directly address the efficiency of write operations or the overall pipeline performance.