Databricks Certified Data Engineer - Associate

Explanation

The correct answer is D. trigger(processingTime="5 seconds").

Why this is correct:

1. Trigger Types in Structured Streaming:

   • ProcessingTime trigger: Executes micro-batches at fixed intervals (e.g., every 5 seconds)
   • Once trigger: Executes only one micro-batch and then stops
   • Continuous trigger: Processes data continuously (not in micro-batches) with millisecond latency

2. Syntax Analysis:

   • trigger("5 seconds") - This is valid but ambiguous in some contexts, though it typically defaults to processingTime
   • trigger(continuous="5 seconds") - Incorrect syntax for continuous trigger (should be trigger(Trigger.Continuous("5 seconds")))
   • trigger(once="5 seconds") - Incorrect syntax for once trigger (should be trigger(Trigger.Once()))
   • trigger(processingTime="5 seconds") - Correct and explicit syntax for processing time trigger

3. Requirements Match:

   • The question specifies "only wants the query to execute a micro-batch to process data every 5 seconds"
   • This exactly matches the processingTime trigger behavior

Detailed Explanation:

In Apache Spark Structured Streaming, triggers control when streaming queries execute micro-batches:

• ProcessingTime trigger: The most common trigger for periodic batch processing. It processes data at fixed intervals regardless of whether new data has arrived.
• Once trigger: Processes all available data once and then stops the query.
• Continuous trigger: Provides low-latency processing (as low as 1ms) but has different semantics and requirements.

For the specific requirement of "every 5 seconds", trigger(processingTime="5 seconds") is the most explicit and correct choice. While trigger("5 seconds") might work in some Spark versions, the explicit processingTime parameter is clearer and less ambiguous.

Alternative Valid Syntax:

In newer Spark versions, you could also use:

• trigger(Trigger.ProcessingTime("5 seconds"))
• trigger(processingTime='5 seconds') (in Python)

But among the given options, trigger(processingTime="5 seconds") is the most precise and correct answer.