What does next-token prediction mean, and why does it produce fluent text?
The model reads the text so far and outputs a probability for every possible next token, picks one, appends it and repeats. Because it was trained on huge amounts of real text, the likely next token is usually grammatical and on topic.
Explain it simply
Think of the autocomplete on your phone keyboard, but trained on a vast amount of text and able to look at everything you have typed so far, not just the last word.
At each step the model produces a score for every token in its vocabulary, turns the scores into probabilities, and chooses one. That token is added to the text and the whole process runs again. A 100-word answer is about 130 or more of these small steps, one after another.
Fluency appears because the training goal, predicting the next token across billions of sentences, forces the model to learn grammar, facts, style and common reasoning patterns along the way. Truth is not part of the goal: a wrong sentence can be very probable if it sounds like text people write.
Example
Prompt so far: "Virat Kohli scored a century in the"
Next-token probabilities (illustrative):
" match" 0.31
" first" 0.18
" final" 0.09
" Test" 0.07
...
Pick one, append it, then predict again.What to say in the interview
"An LLM is an autoregressive model: it outputs a probability distribution over the next token given all previous tokens, samples one, appends it and repeats. Fluency comes from training on large text corpora. But it optimises for plausible continuation, not correctness, which is why hallucinations happen."
Common mistakes
- Saying the model plans the whole answer in advance. It generates one token at a time, though its internal state carries context.