Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
A data engineer and data analyst are working together on a data pipeline. The data engineer is working on the raw, bronze, and silver layers of the pipeline using Python, and the data analyst is working on the gold layer of the pipeline using SQL. The raw source of the pipeline is a streaming input. They now want to migrate their pipeline to use Delta Live Tables. Which of the following changes will need to be made to the pipeline when migrating to Delta Live Tables?
Real Exam
Community
KKeng
A
None of these changes will need to be made
B
The pipeline will need to stop using the medallion-based multi-hop architecture
C
The pipeline will need to be written entirely in SQL
D
The pipeline will need to use a batch source in place of a streaming source
E
The pipeline will need to be written entirely in Python
Explanation:
Explanation
Delta Live Tables (DLT) is designed to be flexible and support various pipeline architectures without requiring major changes:
-
Medallion architecture support: DLT fully supports the medallion (bronze-silver-gold) architecture. In fact, it's a recommended pattern for organizing data quality and transformations.
-
Multi-language support: DLT supports both Python and SQL. The data engineer can continue using Python for the raw, bronze, and silver layers, while the data analyst can continue using SQL for the gold layer.
-
Streaming support: DLT supports both batch and streaming pipelines. The existing streaming source can be maintained without switching to batch processing.
-
No requirement for single language: DLT doesn't require pipelines to be written entirely in Python or SQL - it supports mixed-language pipelines.
Therefore, none of the listed changes are required when migrating to Delta Live Tables. DLT is designed to accommodate existing pipeline patterns and provides additional benefits like automatic data quality monitoring, simplified orchestration, and improved reliability.
Comments
Loading comments...