
Ultimate access to all questions.
In the context of building a data pipeline with Delta Lake on Azure Databricks that involves incremental data processing, consider the following scenario: Your organization requires strict compliance with data governance policies, including the ability to audit all changes made to the data. Additionally, the solution must support debugging efforts by allowing data engineers to trace back to the origin of any data inconsistencies. Given these requirements, what is the significance of tracking the history of table transactions in Delta Lake, and how can you review this history? Choose the best option that addresses both the auditing and debugging needs.
A
Tracking the history of table transactions is optional in Delta Lake, as the system inherently prevents any data inconsistencies, making auditing and debugging unnecessary.
B
Delta Lake automatically tracks all table transactions without any configuration needed, but this feature is only useful for auditing purposes and not for debugging.
C
Tracking the history of table transactions is crucial for both auditing and debugging in Delta Lake, as it allows you to see who made changes, when, and what those changes were, facilitating compliance and issue resolution.
D
Reviewing the history of table transactions in Delta Lake is not supported, requiring external tools to be integrated for auditing and debugging purposes.