Answer-first summary for fast verification
Answer: Stop the Cloud Dataflow pipeline with the Drain option. Create a new Cloud Dataflow job with the updated code
Option D is the correct answer. When dealing with significant changes like a different windowing algorithm and triggering strategy in a streaming Cloud Dataflow pipeline, using the Drain option ensures that the existing pipeline job completes the processing of any in-flight data before stopping. This approach minimizes the risk of data loss. Updating the pipeline in-flight with option A may not handle such major changes reliably, increasing the risk of unpredictable results and potential data loss.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
You are managing a streaming Cloud Dataflow pipeline that processes real-time data. Your engineering team has developed an updated version of this pipeline that includes changes to the windowing algorithm and triggering strategy. Your objective is to deploy this new version to replace the currently running one. It's critical to perform the update in such a way that ensures no data is lost during the transition. What steps should you take to achieve this?
A
Update the Cloud Dataflow pipeline in-flight by passing the --update option with the --jobName set to the existing job name
B
Update the Cloud Dataflow pipeline in-flight by passing the --update option with the --jobName set to a new unique job name
C
Stop the Cloud Dataflow pipeline with the Cancel option. Create a new Cloud Dataflow job with the updated code
D
Stop the Cloud Dataflow pipeline with the Drain option. Create a new Cloud Dataflow job with the updated code