What is the difference between a base model and an instruction-tuned model?
A base model is trained only to continue text, so it may ramble or continue your question with more questions. An instruction-tuned model is further trained on examples of instructions and good answers, so it follows requests and replies helpfully.
Explain it simply
A base model is like a student who has read the entire library but never sat in a class. Ask "What is the capital of Karnataka?" and it may carry on with "What is the capital of Kerala? What is the capital of Goa?" because that is what quiz pages look like. It predicts what text usually comes next.
An instruction-tuned model got extra training after that. It saw many pairs of "instruction, good response", often followed by preference-based training that rewards helpful and safe replies. Now the same question gets "Bengaluru."
The knowledge mostly comes from the first stage. The second stage changes behaviour and format, not how much the model knows.
Example
Prompt: Translate to Hindi: Good morning
Base model: Translate to Hindi: Good evening
Translate to Hindi: Thank you
Instruct model: शुभ प्रभातThe base model treats the prompt as the start of a worksheet and continues the pattern.
What to say in the interview
"A base model only predicts the next token from pretraining data. An instruction-tuned model is fine-tuned on instruction and response pairs, and usually on human preference data, so it follows requests and holds a conversation. Almost every chat product uses an instruction-tuned model; base models are mostly used as a starting point for further tuning."
Common mistakes
- Saying instruction tuning teaches the model new facts. It mainly shapes behaviour.
- Assuming a base model is simply a worse chatbot. It is a different tool, built for continuation.