AWS Certified Cloud Practitioner
Get started today
Ultimate access to all questions.
Deep dive into the quiz with AI chat providers.
We prepare a focused prompt with your quiz and certificate details so each AI can offer a more tailored, in-depth explanation.
Why do Transformers use positional encodings?
Real Exam
Community
RRitesh
A
To increase randomness in generation
B
To provide sequence order information to the model
C
To reduce model size
D
To avoid gradient vanishing
Explanation:
Transformers use positional encodings to provide sequence order information to the model. Unlike recurrent neural networks (RNNs) that process sequences sequentially and inherently capture order, Transformers process all tokens in parallel using self-attention mechanisms. This parallel processing makes them more efficient but loses the inherent sequence order information. Positional encodings are added to the input embeddings to inject information about the relative or absolute position of tokens in the sequence, allowing the model to understand the order of elements in the input sequence.
Comments
Loading comments...