Answer-first summary for fast verification
Answer: All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped.
## Explanation In Delta Live Tables (DLT), when a pipeline is configured to run in **Production mode** with **Continuous Pipeline Mode**, the behavior is as follows: 1. **Continuous Processing**: The pipeline will continuously process data at set intervals until manually stopped. This is different from Triggered mode which runs once and stops. 2. **Compute Resource Management**: In Production mode with Continuous Pipeline Mode, compute resources are deployed specifically for the pipeline execution and are terminated when the pipeline is stopped. This is more cost-effective as resources are not kept idle. 3. **Dataset Types**: The question mentions both streaming datasets (STREAMING LIVE TABLE) and batch datasets (LIVE TABLE). In Continuous mode, streaming datasets will continuously process new data, while batch datasets will be updated according to their refresh schedule. **Why Option C is correct**: - "All datasets will be updated at set intervals until the pipeline is shut down" - This describes continuous processing - "The compute resources will be deployed for the update and terminated when the pipeline is stopped" - This describes the compute lifecycle in Production Continuous mode **Why other options are incorrect**: - **Option A**: Incorrect because compute resources do NOT persist in Production Continuous mode - **Option B**: Incorrect because the pipeline doesn't run just once in Continuous mode - **Option D**: Incorrect because the pipeline doesn't run just once in Continuous mode This configuration is ideal for production workloads where you need continuous data processing with efficient resource utilization.
Author: Keng Suppaseth
Ultimate access to all questions.
No comments yet.
A Delta Live Table pipeline includes two datasets defined using STREAMING LIVE TABLE. Three datasets are defined against Delta Lake table sources using LIVE TABLE. The table is configured to run in Production mode using the Continuous Pipeline Mode. What is the expected outcome after clicking Start to update the pipeline assuming previously unprocessed data exists and all definitions are valid?
A
All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist to allow for additional testing.
B
All datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional testing.
C
All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped.
D
All datasets will be updated once and the pipeline will shut down. The compute resources will be terminated.