Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
You are tasked with implementing a data pipeline that processes streaming data from a sensor network. The data needs to be processed incrementally and deduplicated. Describe the architecture and operations necessary to achieve this using Delta Lake with streaming workloads.
Simulated
A
Use a batch processing approach with Delta Lake to handle incremental processing and deduplication.
B
Implement a streaming architecture with Delta Lake using MERGE INTO for deduplication and auto-compaction for incremental processing.
C
Use a combination of Kafka and Delta Lake for streaming data without handling deduplication.
D
Implement a custom Python script to handle streaming data and deduplication without using Delta Lake.
Comments
Loading comments...