Explanation:
The correct line of code to fill in the blank to execute a micro-batch to process data every 5 seconds is:
D. trigger(processingTime="5 seconds")
Option A ("trigger('5 seconds')") would not work because it does not specify that the trigger should be a processing time trigger, which is necessary to trigger a micro-batch processing at regular intervals.
Option B ("trigger()") would not work because it would use the default trigger, which is not a processing time trigger.
Option C ("trigger(once='5 seconds')") would not work because it would only trigger the query once, not at regular intervals.
Option E ("trigger(continuous='5 seconds')") would not work because it would trigger the query to run continuously, without any pauses in between, which is not what the data engineer wants.
Ultimate access to all questions.
No comments yet.
A data engineer is working on a Structured Streaming job within Databricks. The job reads data from an existing table, processes the data, and then writes the output to a new table in a streaming fashion. The data engineer aims to configure the streaming job so that it processes the data in micro-batches with a frequency of every 5 seconds.
Given the code block used by the data engineer, which line of code should be used to accomplish this 5-second micro-batch interval requirement?
A
trigger("5 seconds")
B
trigger()
C
trigger(once="5 seconds")
D
trigger(processingTime="5 seconds")