Answer-first summary for fast verification
Answer: Batch the job into ten-second increments
Correct Designing a pipeline that processes tens of thousands of messages per second requires careful consideration to avoid overwhelming the system with API calls. Making individual callouts for each element would necessitate handling an equivalent number of API calls per second, leading to significant backpressure if each call takes approximately 1 second. Batching these requests is a more efficient approach, as it reduces the number of API calls and mitigates backpressure. Reference: [Guide to Common Cloud Dataflow Use Case Patterns Part 1](https://cloud.google.com/blog/products/data-analytics/guide-to-common-cloud-dataflow-use-case-patterns-part-1)
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
How can you efficiently design a pipeline to generate a globally unique identifier (GUID) for new website users, utilizing a service that processes data points and returns a GUID? The pipeline must support tens of thousands of messages per second and employ multi-threading to reduce system backpressure.
A
Create a static pipeline in the class definition
B
Batch the job into ten-second increments
C
Use HTTP calls to call the service
D
Create a new object in the startBundle method of DoFn