Answer-first summary for fast verification
Answer: The Dataflow job's watermark isn't advancing because the `eventTimestamp` isn't factored into the windowing logic. Adjust the job's windowing logic to include `eventTimestamp`.
The correct answer is **C**: The Dataflow job's watermark is not advancing due to the `eventTimestamp` attribute being overlooked in the windowing logic. This oversight prevents the system from accurately tracking event time progress, causing delays beyond the acceptable 30-second window for the advertising department. - **Why C is correct**: The watermark's role is to signal when data for a window is considered complete. Ignoring `eventTimestamp` disrupts this, leading to delayed message delivery. - **Why others are incorrect**: - **A**: The issue isn't with message consumption but with processing logic. - **B**: The job's system lag is minimal (5 seconds), indicating processing isn't the bottleneck. - **D**: There's no evidence the web server's message push rate is at fault. To resolve, integrate `eventTimestamp` into the Dataflow job's windowing logic to ensure timely message delivery.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
A web server sends click events to a Pub/Sub topic, with each event having an eventTimestamp attribute marking the click time. A Dataflow streaming job processes these events and sends the results to another Pub/Sub topic for the advertising department. Despite the Dataflow job's system lag being only 5 seconds and data freshness at 40 seconds, the advertising department is not receiving messages within 30 seconds of the click. What's the root cause of this delay, and how can it be resolved?
A
The advertising department is slow in consuming messages. Collaborate with them to improve consumption speed.
B
The Dataflow job processes messages in under 30 seconds but struggles with the Pub/Sub subscription backlog. Optimize the job or scale up workers to address this.
C
The Dataflow job's watermark isn't advancing because the eventTimestamp isn't factored into the windowing logic. Adjust the job's windowing logic to include eventTimestamp.
D
The web server's message push rate to Pub/Sub is insufficient. Coordinate with the web server team to enhance the push rate.
No comments yet.