Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
In the context of optimizing a data pipeline that processes frequent updates and deletions in a Delta table, consider the following scenario: Your organization is prioritizing cost-efficiency and performance while ensuring data consistency and compliance with audit requirements. Which Delta Lake feature should you leverage to meet these objectives, and why? Choose the best option from the following:
A
Implement a custom solution that bypasses Delta Lake's transaction log to directly modify data files, aiming to reduce logging overhead and improve performance.
B
Utilize Delta Lake's transaction log exclusively for tracking changes, performing updates and deletions as separate transactions to ensure data consistency.
C
Employ Delta Lake's upsert capability to efficiently update or insert data based on a unique key, leveraging the transaction log for consistency without significant performance overhead.
D
Apply Delta Lake's delete feature in isolation to remove data based on specific conditions, relying on the transaction log for audit trails.