AWS Certified Cloud Practitioner

Explanation

Correct Answer: A (Increase top-p from 0.7 to 0.9)

Why this is correct:

1. Top-p (nucleus sampling) controls the diversity of token selection by considering only the smallest set of tokens whose cumulative probability exceeds the threshold p.
2. Increasing top-p from 0.7 to 0.9 means the model considers a larger pool of tokens (90% of the probability mass instead of 70%), allowing for more exploration and diversity in the generated text.
3. This helps address the issue of the model ignoring key details and producing too-short answers because it allows the model to consider more possible continuations, including those that might contain important details.

Why other options are incorrect:

• B (Decrease temperature to 0.1): Lower temperature makes the model more deterministic and conservative, which would likely make the problem worse by making the model even more likely to stick to the most probable tokens and produce shorter, more generic responses.

• C (Set top-k from 100 to 10): Decreasing top-k from 100 to 10 restricts the model to consider only the top 10 most likely tokens, which reduces exploration and diversity, making the problem worse.

• D (Reduce top-p from 0.9 to 0.4): Decreasing top-p restricts the token pool to a smaller set (only 40% of the probability mass), which reduces exploration and diversity, making the problem worse.

Key Concept:

• Top-p (nucleus sampling): Higher values = more diversity/exploration
• Temperature: Higher values = more randomness/creativity
• Top-k: Higher values = more token options considered

For the specific problem of "ignoring key details and producing too-short answers," increasing top-p is the appropriate adjustment to encourage more exploration and potentially longer, more detailed responses.