What do repetition, frequency and presence penalties do during generation?
They lower the chance of tokens that have already appeared. A frequency penalty grows with each repeat, a presence penalty applies once a token has appeared at all. They reduce loops and repeated phrases but can hurt text that needs repetition.
Explain it simply
Imagine a friend who keeps saying "basically" in every sentence. You start a game: each time they say it, they lose a point. Soon they use other words.
Penalties work like that on token scores. A frequency penalty subtracts more the more times a token has already been used, discouraging heavy repetition. A presence penalty subtracts a flat amount once the token has appeared, nudging the model towards new topics. Some systems expose a single repetition penalty that multiplies down the score of seen tokens.
They help when output gets stuck in loops, which happens often at low temperature. But strong values backfire: a table of numbers, code with repeated keywords, or a name that must appear several times can become awkward or wrong. Keep values small and test.
Example
Without penalty: "The app is fast. The app is simple. The app is ..."
With penalty: "The app is fast, simple and easy to learn."What to say in the interview
"Penalties down-weight tokens already generated, to reduce repetition: frequency scales with count, presence is a one-time nudge. I use small values only when I see loops, since they can distort code, lists and names that legitimately repeat."