You are tasked with implementing a data pipeline that processes real-time data from a IoT devices. The data needs to be processed incrementally and deduplicated. Describe the architecture and operations necessary to achieve this using Delta Lake with streaming workloads.
