Google Professional Machine Learning Engineer
Get started today
Ultimate access to all questions.
You recently developed a deep learning model using the Keras API. Initially, you trained the model using a single GPU, but found the training process to be too slow. To address this, you then used TensorFlow's tf.distribute.MirroredStrategy to distribute the training across 4 GPUs without making any other changes. Despite this, you did not observe any decrease in training time. Considering this, what should you do next to effectively improve the training time?
You recently developed a deep learning model using the Keras API. Initially, you trained the model using a single GPU, but found the training process to be too slow. To address this, you then used TensorFlow's tf.distribute.MirroredStrategy to distribute the training across 4 GPUs without making any other changes. Despite this, you did not observe any decrease in training time. Considering this, what should you do next to effectively improve the training time?
Explanation:
The correct answer is D: Increase the batch size. When using tf.distribute.MirroredStrategy, TensorFlow automatically handles the distribution of the dataset across the GPUs. However, to effectively utilize the additional computational resources provided by multiple GPUs, you should increase the global batch size. This ensures that each GPU receives a larger batch of data to process, resulting in more efficient utilization of the GPUs and potentially faster training times. The global batch size should typically be the batch size for a single GPU multiplied by the number of GPUs. For example, if the initial batch size for a single GPU was 64, the global batch size for 4 GPUs should be 256 (64 * 4).