What happens when you set temperature to 0?
The model effectively always picks the most likely next token, so output becomes close to deterministic and repeatable. It suits extraction and classification, but it is not a guarantee of a correct or perfectly identical answer.
Explain it simply
Temperature divides the model's raw scores before they become probabilities. Lower values sharpen the distribution so the top choice dominates. At zero, the system skips the dice roll and just takes the top token.
That is what you want when the job has one right shape of answer: pulling a date out of an email, labelling a ticket, or producing JSON. You get stable output across runs, which helps testing.
Limits to remember. A confident wrong answer stays wrong, because the most likely token is not always the true one. Tiny numerical differences on different hardware or batch conditions can still change a close call, so exact repeatability is not promised. And zero temperature can make creative writing repetitive.
Example
Task: "Extract the invoice number from the text below."
Temperature 0 -> "INV-2043" on every run
Temperature 1.2 -> sometimes "INV-2043", sometimes a rephrased sentenceWhat to say in the interview
"Temperature zero means greedy-like selection, giving near-deterministic output, ideal for extraction, classification and tests. It does not improve correctness by itself, and I would still validate the result."
Common mistakes
- Assuming low temperature reduces hallucination. It only reduces randomness.