AWS Certified AI Practitioner

Explanation

Temperature and top-p (nucleus sampling) are both hyperparameters that control the randomness and creativity of language model outputs in Amazon Bedrock:

Temperature:

• Controls the randomness of predictions by scaling the logits before applying softmax
• Higher temperature (e.g., 0.8-1.0) increases randomness and creativity
• Lower temperature (e.g., 0.1-0.3) makes outputs more deterministic and focused

Top-p (Nucleus Sampling):

• Filters the vocabulary to the smallest set of tokens whose cumulative probability exceeds p
• Only considers tokens from this "nucleus" for sampling
• Helps maintain coherence while allowing diversity

How They Work Together:

1. Complementary Controls: Both parameters influence output diversity and creativity
2. Sequential Application: Temperature is applied first to scale logits, then top-p filters the probability distribution
3. Tuning Strategy: They can be tuned together to balance creativity and coherence
4. Optimal Output Diversity: The combination allows fine-grained control over how "creative" vs. "focused" the model should be

Why Other Options Are Incorrect:

• B: Top-p does not affect latency only; it's a sampling parameter that affects output quality
• C: Temperature doesn't control grammar, and top-p doesn't control sentence length
• D: Neither parameter reduces vocabulary size; they control sampling from the existing vocabulary

Best Practice:

For most use cases, start with moderate values (temperature ~0.7, top-p ~0.9) and adjust based on the desired balance between creativity and coherence for your specific application.