Ultimate access to all questions.
Answer-first summary for fast verification
Answer: trigger(availableNow=True)
To understand the correct answer, let's explore Trigger Intervals in Structured Streaming. The `trigger` method determines when the system should process the next set of data. Triggers control the frequency of micro-batches, with Spark automatically detecting and processing new data since the last trigger by default. The `Trigger.AvailableNow` option, introduced in DBR 10.1 for Scala and DBR 10.2 for Python and Scala, is designed to process all available data in micro-batches and then stop, making `trigger(availableNow=True)` the correct choice for this scenario.
Author: LeetQuiz Editorial Team
No comments yet.
A data engineer has set up a Structured Streaming job to read from a table, manipulate the data, and then perform a streaming write into a new table. The provided code block is missing a crucial line to complete the operation. The goal is to have the query execute in multiple micro-batches, process all available data, and then stop automatically. Which of the following code lines should fill the blank to achieve this?
A
trigger(processingTime="500ms")
B
trigger(availableNow=True)
C
trigger(once=True)
D
trigger(processingTime="now")
E
trigger(available_Now="True")