Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
A Delta Live Table pipeline includes two datasets defined using STREAMING LIVE TABLE. Three datasets are defined against Delta Lake table sources using LIVE TABLE. The table is configured to run in Production mode using the Continuous Pipeline Mode. Assuming previously unprocessed data exists and all definitions are valid, what is the expected outcome after clicking Start to update the pipeline?
Real Exam
Community
KKeng
A
All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist to allow for additional testing.
B
All datasets will be updated once and the pipeline will persist without any processing. The compute resources will persist but go unused.
C
All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped.
D
All datasets will be updated once and the pipeline will shut down. The compute resources will be terminated.
E
All datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional testing.
Explanation:
Explanation
In Delta Live Tables (DLT), when a pipeline is configured to run in Production mode using Continuous Pipeline Mode, the expected behavior is:
-
Continuous Processing: The pipeline will continuously process data at set intervals until explicitly stopped. This is the key characteristic of Continuous mode.
-
Resource Management: In Production mode, compute resources are deployed for the duration of the pipeline run and terminated when the pipeline is stopped. This differs from Development mode where resources persist for additional testing.
-
Dataset Types: The question mentions both streaming datasets (STREAMING LIVE TABLE) and batch datasets (LIVE TABLE against Delta Lake sources). In Continuous mode, streaming datasets will process data continuously, while batch datasets will be updated according to their refresh schedule.
Why other options are incorrect:
- A: Incorrect because while it correctly states datasets will be updated at intervals, it incorrectly says compute resources persist for additional testing (that's Development mode behavior).
- B: Incorrect because Continuous mode doesn't update just once; it processes continuously.
- D: Incorrect because Continuous mode doesn't shut down after one update; it runs continuously.
- E: Incorrect because Continuous mode doesn't update just once, and compute resources don't persist in Production mode.
Key Concepts:
- Production Mode: Optimized for cost efficiency, resources are terminated when pipeline stops
- Continuous Pipeline Mode: Processes data continuously at set intervals
- Development Mode: Resources persist for testing and development purposes
Comments
Loading comments...