For a DLT pipeline that needs to be triggered based on complex event conditions from multiple data sources, which method best supports the dynamic initiation of pipeline runs?

Real Exam

Rely on manual initiation of pipeline runs based on reports from data source administrators.

2.7%

Configure the DLT pipeline to poll data sources at regular intervals, triggering runs based on data presence.

15.1%

Use external event management systems like Apache Kafka to aggregate events, invoking the DLT pipeline via the Databricks REST API when conditions are met.

69.9%

Implement a custom Spark Structured Streaming application that listens for specific events and triggers DLT pipeline runs via REST API calls.

12.3%

Databricks Certified Data Engineer - Professional

Get started today

Comments

For a DLT pipeline that needs to be triggered based on complex event conditions from multiple data sources, which method best supports the dynamic initiation of pipeline runs?