Answer-first summary for fast verification
Answer: Send the request again with a smaller batch of instances.
The 'Out of Memory' error during an online prediction request on Vertex AI indicates that the amount of data being sent in a single request is too large. To resolve this issue, you can send the request again with a smaller batch of instances. This reduces the memory footprint of the request, potentially avoiding the out-of-memory error. This is a more immediate solution compared to the other options.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
You have deployed a machine learning model on Google's Vertex AI platform for real-time inference. During an online prediction request, you encounter an 'Out of Memory' error, indicating that the prediction node lacks sufficient memory to process the request. What should you do to resolve this issue?
A
Use batch prediction mode instead of online mode.
B
Send the request again with a smaller batch of instances.
C
Use base64 to encode your data before using it for prediction.
D
Apply for a quota increase for the number of prediction requests.
No comments yet.