Google Professional Data Engineer

Explanation

This question addresses performance optimization in Google Cloud Bigtable when dealing with large-scale data ingestion and processing.

Key Issues:

• 10 TB initial data load with suboptimal read/write performance
• Data rapidly growing every hour during 30-day campaign
• Need to improve performance while minimizing cost

Analysis of Options:

Option A (Correct): Redefining the schema to evenly distribute reads and writes across the row space is the optimal solution. In Bigtable, performance is heavily dependent on row key design. When reads and writes are concentrated on a small number of rows ("hotspotting"), it creates bottlenecks. By distributing the workload evenly across the row space, you can leverage Bigtable's automatic sharding and achieve better parallelism.

Option B (Incorrect): Simply waiting for the cluster size to increase won't resolve the fundamental schema design issue. Hotspotting will persist regardless of cluster size, and this approach would increase costs without solving the performance problem.

Option C (Incorrect): Using a single row key for frequently updated values would actually worsen the hotspotting problem. This concentrates all updates on one row, creating a severe bottleneck.

Option D (Incorrect): Using sequentially increasing numeric IDs can create hotspotting because new data always goes to the "end" of the table, concentrating writes on a small number of tablets.

Best Practice: The optimal approach is to design row keys that distribute the workload evenly, such as using hash prefixes, salting, or other techniques that spread operations across the entire row space.