Answer-first summary for fast verification
Answer: They can navigate to the DLT pipeline page, click on each table, and view the data quality statistics.
To identify the table in a Delta Live Tables (DLT) pipeline where data is being dropped due to quality concerns, the data engineer can navigate to the DLT pipeline page, click on each table in the pipeline, and view the data quality statistics. These statistics often include information about records dropped, violations of expectations, and other data quality metrics. By examining the data quality statistics for each table in the pipeline, the data engineer can determine at which table the data is being dropped. **Why other options are incorrect:** - **A**: Setting up separate expectations for each table helps define quality rules but doesn't directly help identify which table is currently dropping records. - **B**: This is incorrect because there is a way to determine which table is dropping records through the DLT UI. - **C**: Email notifications can alert about quality issues but don't specifically identify which table is dropping records. - **E**: The "Error" button typically shows pipeline execution errors, not detailed data quality statistics for individual tables.
Author: Keng Suppaseth
Ultimate access to all questions.
No comments yet.
A data engineer has three tables in a Delta Live Tables (DLT) pipeline. They have configured the pipeline to drop invalid records at each table. They notice that some data is being dropped due to quality concerns at some point in the DLT pipeline. They would like to determine at which table in their pipeline the data is being dropped.
Which of the following approaches can the data engineer take to identify the table that is dropping the records?
A
They can set up separate expectations for each table when developing their DLT pipeline.
B
They cannot determine which table is dropping the records.
C
They can set up DLT to notify them via email when records are dropped.
D
They can navigate to the DLT pipeline page, click on each table, and view the data quality statistics.
E
They can navigate to the DLT pipeline page, click on the "Error" button, and review the present errors.