Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
A Delta Live Table pipeline includes two datasets defined using STREAMING LIVE TABLE. Three datasets are defined against Delta Lake table sources using LIVE TABLE. The table is configured to run in Development mode using the Continuous Pipeline Mode. Assuming previously unprocessed data exists and all definitions are valid, what is the expected outcome after clicking Start to update the pipeline?
Real Exam
Community
KKeng
A
All datasets will be updated once and the pipeline will shut down. The compute resources will be terminated.
B
All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist until the pipeline is shut down.
C
All datasets will be updated once and the pipeline will persist without any processing. The compute resources will persist but go unused.
D
All datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional testing.
Explanation:
Explanation
In Delta Live Tables (DLT), when a pipeline is configured to run in Development mode with Continuous Pipeline Mode, the behavior is as follows:
-
Development Mode: This mode is designed for testing and development purposes. When you start a pipeline in development mode, it processes all available data once and then shuts down.
-
Continuous Pipeline Mode: This refers to the pipeline's ability to continuously process streaming data. However, when combined with Development mode, the pipeline will:
- Process all datasets (both streaming and batch) once
- Then automatically shut down
- Compute resources will be terminated to avoid unnecessary costs
-
Key Distinctions:
- Production Mode: Would continue running and processing data at set intervals
- Development Mode: Processes once and stops, which is ideal for testing pipeline logic without incurring ongoing compute costs
-
Dataset Types: The question mentions both streaming datasets (STREAMING LIVE TABLE) and batch datasets (LIVE TABLE). In Development mode, both types will be processed once with the available data.
Why Option A is correct:
- All datasets are updated once (true for Development mode)
- The pipeline shuts down (characteristic of Development mode)
- Compute resources are terminated (to save costs in development environment)
Why other options are incorrect:
- Option B: Describes Production mode behavior, not Development mode
- Option C: Incorrect because the pipeline doesn't persist after processing in Development mode
- Option D: Incorrect because compute resources are terminated, not persisted
This behavior allows developers to test their pipeline logic efficiently without leaving resources running unnecessarily.
Comments
Loading comments...