AWS Certified Cloud Practitioner

Explanation

Correct Answer: B (Use nucleus sampling by setting top-p = 0.8)

Why this is correct:

1. Nucleus sampling (top-p sampling) is specifically designed to sample from the smallest set of tokens whose cumulative probability exceeds a threshold (p). This means the model dynamically selects tokens based on cumulative probability rather than a fixed number of tokens.

2. How top-p works:

   • Tokens are sorted by probability in descending order

   • The algorithm adds tokens to the selection set until the cumulative probability exceeds the top-p value (e.g., 0.8)

   • Sampling then occurs only from this dynamically determined set

   • This ensures balanced diversity by considering probability distributions rather than fixed rankings

Why other options are incorrect:

A. Temperature = 0.0 - This makes the model deterministic by always selecting the highest probability token (no diversity).

C. Lower top-k to 5 - This restricts sampling to only the top 5 tokens by probability, which doesn't use cumulative probability and may exclude relevant tokens with lower individual probabilities but significant cumulative contribution.

D. Increase temperature to 2.0 - While temperature affects randomness, it doesn't specifically sample based on cumulative probability. Higher temperature makes the probability distribution more uniform, increasing randomness but not using the cumulative probability approach requested.

Key takeaway: Top-p (nucleus) sampling is the technique specifically designed for sampling from tokens based on cumulative probability thresholds, providing balanced diversity while maintaining coherence.