AWS Certified Cloud Practitioner

Explanation

Repetition penalty is a specific inference setting designed to reduce repetitive outputs from language models. Here's why option B is correct:

What is Repetition Penalty?

• Repetition penalty is a parameter that penalizes tokens that have already appeared in the generated text

• A value greater than 1.0 (like 1.2) discourages repetition

• A value less than 1.0 encourages repetition

• The penalty is applied to the probability of tokens that have already been generated

Why Other Options Are Incorrect:

A. Increase temperature - Temperature controls randomness/creativity:

• Higher temperature (e.g., 0.8-1.0) increases randomness and diversity

• Lower temperature (e.g., 0.1-0.3) makes outputs more deterministic and focused

• While higher temperature can sometimes reduce repetition, it's not specifically designed for this purpose and can make outputs less coherent

C. Reduce top-p - Top-p (nucleus sampling):

• Controls the cumulative probability threshold for token selection

• Lower top-p values make outputs more focused but don't specifically target repetition

• This affects which tokens are considered, not whether they've been used before

D. Reduce top-k - Top-k sampling:

• Limits the number of tokens considered to the top k most likely tokens

• Lower top-k values restrict diversity but don't specifically prevent repetition

• This affects token selection pool size, not repetition patterns

Practical Application:

• When you notice a model repeating phrases like "the the the" or "I think I think"

• Set repetition penalty to values like 1.1-1.3 to moderately discourage repetition

• Values like 1.5+ can strongly discourage repetition but may affect coherence

• This is particularly useful for long-form generation, creative writing, or dialogue systems where natural flow is important

Best Practice:

For reducing repetition while maintaining coherence, use a combination of:

1. Repetition penalty (1.1-1.2) to specifically target repeated tokens

2. Moderate temperature (0.7-0.9) for balanced creativity

3. Appropriate top-p (0.9-0.95) for focused yet diverse outputs