Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
A Delta Live Table pipeline includes two datasets defined using STREAMING LIVE TABLE. Three datasets are defined against Delta Lake table sources using LIVE TABLE. The table is configured to run in Development mode using the Continuous Pipeline Mode.
Assuming previously unprocessed data exists and all definitions are valid, what is the expected outcome after clicking Start to update the pipeline?
Real Exam
Community
KKeng
A
All datasets will be updated once and the pipeline will shut down. The compute resources will be terminated.
B
All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist until the pipeline is shut down.
C
All datasets will be updated once and the pipeline will persist without any processing. The compute resources will persist but go unused.
D
All datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional testing.
E
All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist to allow for additional testing.
Explanation:
Explanation
Correct Answer: E
This question tests understanding of Delta Live Tables (DLT) pipeline behavior in Development mode with Continuous Pipeline Mode.
Key Concepts:
-
Development Mode:
- Reuses clusters to avoid restart overhead
- Clusters run for 2 hours by default (configurable via
pipelines.clusterShutdown.delay) - Disables pipeline retries for immediate error detection
-
Continuous Pipeline Mode:
- Continuously processes new data as it arrives
- Updates datasets at set intervals
- Pipeline runs until manually shut down
-
Dataset Types:
STREAMING LIVE TABLE: Processes streaming data continuouslyLIVE TABLE: Processes batch data
Why Option E is Correct:
- "All datasets will be updated at set intervals": In Continuous Pipeline Mode, DLT continuously processes data at regular intervals
- "until the pipeline is shut down": The pipeline runs continuously until manually stopped
- "The compute resources will persist to allow for additional testing": In Development mode, clusters persist (default 2 hours) to facilitate testing and debugging
Why Other Options are Incorrect:
- A: Incorrect - Pipeline doesn't shut down after one update in Continuous mode
- B: Partially correct about updates, but incorrect about compute persistence - in Development mode, clusters persist for testing
- C: Incorrect - Pipeline continues processing at intervals, doesn't just persist idle
- D: Incorrect - Pipeline doesn't shut down after one update
Additional Context:
The provided text confirms that in Development mode, Delta Live Tables "reuses a cluster to avoid the overhead of restarts" and clusters "run for two hours when development mode is enabled," supporting the persistence of compute resources for testing purposes.
Comments
Loading comments...