Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
In the context of managing a data pipeline that processes sensitive customer information, you are tasked with ensuring that updates and deletions in a Delta table are performed atomically and consistently to maintain data integrity and comply with GDPR requirements. The solution must also optimize for cost and performance. Given these constraints, which of the following approaches best leverages Delta Lake's features to meet these requirements? Choose the best option.
A
Implement a custom solution that manually tracks changes in an external log and applies updates and deletions in batch processes to minimize costs.
B
Disable Delta Lake's transaction log to reduce storage costs and perform direct file operations for updates and deletions, accepting the risk of potential inconsistencies.
C
Utilize Delta Lake's upsert feature for atomic updates and inserts based on a unique key, and the DELETE command for atomic deletions, ensuring compliance and data integrity without sacrificing performance.
D
Rely on Delta Lake's automatic compaction feature to periodically reorganize data files, indirectly managing updates and deletions through file consolidation.