Google Professional Data Engineer

Explanation

Option C is the correct answer because:

• Try-catch block in DoFn: This handles errors during data transformation gracefully without stopping the entire pipeline
• Write erroneous rows to PubSub directly: This ensures that all failing data is captured and stored in a durable message queue for later reprocessing
• Reprocessing capability: PubSub allows you to replay messages, enabling reprocessing of failed data
• Reliability: The pipeline continues processing valid data while capturing errors separately

Why other options are less optimal:

• Option A: Filtering skips errors but doesn't capture them for reprocessing
• Option B: Extracting from logs is unreliable and difficult to reprocess systematically
• Option D: Using sideOutputs is good for error handling, but storing to PubSub "later" adds complexity and potential data loss

This approach ensures pipeline reliability while maintaining the ability to reprocess all failing data efficiently.