Answer-first summary for fast verification
Answer: Use Delta Lake for ACID transactions and time travel features combined with Databricks Unity Catalog for data governance and lineage.
Delta Lake is an open-source storage layer that brings ACID transactions to Apache Spark and big data workloads, ensuring all data modifications are atomic, consistent, isolated, and durable. It also offers time travel features for accessing and reverting to previous data versions, crucial for auditability. Databricks Unity Catalog provides centralized metadata management, offering data governance and lineage capabilities. Together, they ensure data integrity, reliability, and compliance with regulatory requirements in the financial services industry.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
In the context of financial services, ensuring strict data lineage and auditability for all processed data within your Databricks lakehouse is crucial. Which combination of Databricks tools and features would best meet these requirements?
A
Implement custom logging within Spark jobs to track data modifications manually.
B
Rely solely on Azure Purview for managing data lineage without integrating with Databricks features.
C
Use Delta Lake for ACID transactions and time travel features combined with Databricks Unity Catalog for data governance and lineage.
D
Utilize MLflow for tracking model versions and manual tagging for dataset versions to simulate data lineage.
No comments yet.