Google Professional Data Engineer
Get started today
Ultimate access to all questions.
NO.28 You are building a report-only data warehouse where the data is streamed into BigQuery via the streaming API. Following Google's best practices, you have both a staging and a production table for the data. How should you design your data loading to ensure that there is only one master dataset without affecting performance on either the ingestion or reporting pieces?
Real Exam
Community
LLeetQuiz
Explanation:
Explanation
Option D is correct because:
- 30-minute intervals: This provides a good balance between data freshness and performance
- Move and delete approach: Moving data from staging to production ensures there's only one master dataset
- Performance optimization: Frequent transfers (every 30 minutes) prevent staging table from growing too large, maintaining good ingestion performance
- Reporting performance: Production table remains clean and optimized for queries
Why other options are incorrect:
- Options A & B (3 hours, 90 minutes): These intervals are too long, causing staging tables to become large and potentially impacting ingestion performance
- Option C (3 hours with move/delete): While better than A/B, 3 hours is still too infrequent for optimal performance
This approach follows BigQuery best practices for streaming data with staging/production tables, ensuring both ingestion and query performance are maintained.
Comments
Loading comments...