Explanation:
Correct Option: C. Auto-scaling
Auto-scaling is a critical feature of Google Cloud Dataflow that automatically adjusts the number of workers in a pipeline based on the current workload. This ensures that the pipeline can handle increases in data volume or complexity without manual intervention, optimizing both performance and cost. Auto-scaling is particularly beneficial for projects with variable workloads, as it provides the flexibility to scale resources up or down as needed.
Why other options are incorrect:
Ultimate access to all questions.
No comments yet.
In the context of optimizing a data processing pipeline on Google Cloud Dataflow for a project that handles highly variable workloads, which feature should be leveraged to ensure that the pipeline automatically adjusts its computational resources to match the current workload demands, thereby optimizing cost and performance? Choose the best option.
A
Load balancing, which evenly distributes tasks across available workers but does not adjust the number of workers based on workload.
B
Data sharding, a technique for partitioning data to improve processing efficiency, unrelated to dynamic resource adjustment.
C
Auto-scaling, which dynamically adjusts the number of workers in response to the workload, ensuring efficient resource utilization.
D
Caching, which stores frequently accessed data in memory to speed up processing but does not adjust resources based on workload.