What is the difference between encoder-only, decoder-only and encoder-decoder models?
Encoder-only models like BERT read the whole input in both directions and suit classification and embeddings. Decoder-only models generate text left to right and power most chat LLMs. Encoder-decoder models read an input then generate an output, as in translation.
Explain it simply
Think of three kinds of students. The first reads a passage fully, then answers "is this positive or negative?" That is an encoder-only model: it sees every token with context from both sides.
The second writes an essay one word at a time, looking only at what has been written so far. That is a decoder-only model, trained on next-token prediction. Most modern chat LLMs are of this type.
The third reads a source sentence completely, then writes the translation, glancing back at the source while writing. That is an encoder-decoder model, used for translation and summarisation.
The choice follows the job: embeddings and classification favour encoders, open-ended generation favours decoders, and input-to-output transformation fits encoder-decoder, although large decoders now do all three by prompting.
Example
Encoder-only: "Great delivery!" -> label: positive
Decoder-only: "Write a poem about monsoon" -> generated text
Encoder-decoder: English sentence -> Hindi sentenceWhat to say in the interview
"Encoders use bidirectional attention and produce representations, decoders use causal attention and generate, and encoder-decoders combine both with cross-attention. Chat LLMs are mostly decoder-only, while many embedding and classification models are encoders."