
Ultimate access to all questions.
Explanation:
In Large Language Models (LLMs), text is processed by breaking it down into smaller units called tokens. Tokens can be words, subwords, or even characters, depending on the tokenization strategy used by the model.
Key points:
The correct answer is B. Tokens because tokens are the fundamental sub-units that LLMs use to process and understand text.
When an LLM processes the word "unbelievable", it splits it into sub-units before encoding. What are these sub-units called?
A
Characters
B
Tokens
C
Embeddings
D
Layers
No comments yet.