What is a large language model (LLM), and how does it generate text?
An LLM is a neural network trained on a huge amount of text to predict the next token. It generates text by predicting one token at a time, adding it to the input, and repeating until it stops.
Explain it simply
Think of the autocomplete on your phone keyboard. You type "Happy" and it suggests "birthday". It has seen that pattern many times, so it guesses the most likely next word. A large language model does the same thing, but it has learned from a far larger amount of text and can keep track of a much longer passage before making its guess.
That is really the core idea. An LLM does not look up an answer in a database. It looks at the text so far and predicts what piece of text should come next. Then it adds that piece to the text and predicts again. A full paragraph is just this loop repeated many times.
"Large" refers to the number of parameters, which are the numbers inside the network that get adjusted during training. More parameters and more training data let the model capture more patterns of grammar, facts and reasoning styles.
Example
Suppose the input is "The capital of India is". The model works roughly like this:
Input: "The capital of India is"
Step 1: predict next token -> " New" (highest probability)
Input: "The capital of India is New"
Step 2: predict next token -> " Delhi"
Input: "The capital of India is New Delhi"
Step 3: predict next token -> "."
Step 4: predict an end-of-text token -> stopAt each step the model produces a probability for every token in its vocabulary. A decoding rule (always pick the top one, or sample with some randomness) chooses which token to add.
How it works
There are two separate phases.
- Training. The model reads a very large collection of text and repeatedly tries to predict the next token. Each wrong guess produces an error signal, and the parameters are nudged to make the right token more likely next time. This is slow and expensive and happens once (plus later updates). Most chat assistants then get further training on instruction-and-answer examples and human feedback so that they follow instructions and respond helpfully.
- Inference. This is what happens when you use the model. The parameters are fixed. Your prompt is split into tokens, passed through the network, and the model generates output tokens one by one. Nothing is learned from your prompt during inference; the model only uses it as context.
Most modern LLMs use the transformer architecture. Its key part, attention, lets each token look at every earlier token in the input and decide which ones matter for the next prediction.
Real-world use
- A customer-support chatbot for a bank that drafts replies to common questions.
- Summarising long documents, such as meeting notes or policy PDFs.
- Code assistants that suggest the next few lines in an editor.
- Translating or rewriting text in a simpler tone.
Common mistakes
- Saying the model "searches the internet" or "looks up facts". A plain LLM only predicts tokens from what it learned; search has to be added as a separate tool.
- Saying the model learns from every chat. During inference the weights do not change.
- Treating fluent output as correct output. The model optimises for likely text, which is not always true text.
What to say in the interview
"An LLM is a neural network, usually a transformer, trained on a large amount of text to predict the next token. At inference time it takes my prompt, predicts a probability for every possible next token, picks one, appends it, and repeats until it produces a stop token. Training is when the weights are learned; inference is when those fixed weights are used to generate. Because it is predicting likely text rather than looking things up, it can sound confident and still be wrong."
Follow-up questions
- What is a token, and why do models work with tokens instead of words?
- What is the difference between a base model and an instruction-tuned model?
- Why can the same prompt give different answers on different runs?
Go deeper
Read these once the basics above feel easy.
Instruction tuning and preference training do not change the basic mechanism. The model is still predicting next tokens, but it has been trained on examples where the "likely next text" is a helpful answer to a question. That is why a base model given "What is 2 + 2?" might continue with more quiz questions, while an instruction-tuned model answers "4".