Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
In a Delta Live Tables (DLT) pipeline, a data engineer is working with three tables and has set up the pipeline to drop invalid records at each stage. The engineer observes that some records are being dropped at some point within this pipeline due to quality issues. To determine the specific table at which these records are being dropped, what approach can the data engineer take?
In a Delta Live Tables (DLT) pipeline, a data engineer is working with three tables and has set up the pipeline to drop invalid records at each stage. The engineer observes that some records are being dropped at some point within this pipeline due to quality issues. To determine the specific table at which these records are being dropped, what approach can the data engineer take?
Explanation:
To identify the table that is dropping the records due to quality concerns in a Delta Live Tables (DLT) pipeline, the data engineer can navigate to the DLT pipeline page, click on each table, and view the data quality statistics. This will provide the necessary insights to determine at which table the data is being dropped.