Databricks Certified Data Engineer - Professional
Comments
Loading comments...
Ultimate access to all questions.
In a real-world scenario, your organization is using Delta Lake to manage data pipelines that process streaming data from various sources. The schema of these sources is subject to frequent changes due to evolving business requirements. You are tasked with ensuring that the data pipeline can handle these schema changes efficiently without requiring downtime or significant rework. Considering the need for scalability, cost-effectiveness, and minimal operational overhead, which of the following approaches best leverages Delta Lake's capabilities to handle schema evolution? (Choose one option)
A
Schema evolution is not supported in Delta Lake, necessitating the creation of a new pipeline for each schema change, which increases operational complexity and cost.
B
Implementing custom application logic to manually detect schema changes and adjust the data pipeline accordingly, which requires extensive development effort and maintenance.
C
Utilizing Delta Lake's native support for schema evolution to automatically accommodate schema changes, such as adding new columns, without impacting existing data or requiring pipeline modifications.
D
Creating a new Delta Lake table with the updated schema for each change and manually migrating data from the old table, which is time-consuming and inefficient.