AWS Certified Cloud Practitioner
Get started today
Ultimate access to all questions.
Deep dive into the quiz with AI chat providers.
We prepare a focused prompt with your quiz and certificate details so each AI can offer a more tailored, in-depth explanation.
A company wants to assess the costs that are associated with using a large language model (LLM) to generate inferences. The company wants to use Amazon Bedrock to build generative AI applications. Which factor will drive the inference costs?
Exam-Like
Community
RRitesh
A
Number of tokens consumed
B
Temperature value
C
Amount of data used to train the LLM
D
Total training time
Explanation:
Explanation
For Amazon Bedrock, inference costs are primarily driven by the number of tokens consumed. Here's why:
How Amazon Bedrock Pricing Works:
- Token-based pricing: Amazon Bedrock charges based on the number of input and output tokens processed during inference.
- Input tokens: The text you provide to the model (prompt)
- Output tokens: The text generated by the model (response)
Why Other Options Are Incorrect:
- B. Temperature value: Temperature is a parameter that controls the randomness/creativity of the model's output (0-1 scale), but it doesn't directly affect pricing.
- C. Amount of data used to train the LLM: This is a fixed cost incurred by the model provider during training, not a variable cost for inference.
- D. Total training time: Similar to option C, this is a one-time training cost, not an ongoing inference cost.
Key Points:
- Different foundation models on Amazon Bedrock have different per-token pricing
- Pricing may vary between input and output tokens
- Some models may have minimum charges or different pricing tiers
- The number of tokens directly correlates with the computational resources required to generate the inference
This token-based pricing model is common across many LLM services as it accurately reflects the actual computational work performed by the model.
Comments
Loading comments...