What is top-k sampling, and how does it differ from top-p?
Top-k keeps only the k most likely tokens and samples among them. Top-p keeps the smallest set of tokens whose probabilities add up to p. Top-k uses a fixed count, while top-p adapts the candidate list to how confident the model is.
Explain it simply
You are choosing a restaurant from a ranked list. Top-k says "only look at the first 5, always". Top-p says "keep reading from the top until together they cover 90% of the votes".
When the model is very sure, say the next token has 95% probability, top-p 0.9 keeps just that one token, while top-k 5 still lets four unlikely tokens in. When the model is unsure and many tokens share the probability, top-p keeps a long list while top-k cuts it at five and may throw away good options.
That adaptivity is why top-p is more common as the main control. Both methods trim the unlikely tail, so a wild token is much less likely to be chosen.
Example
Probabilities: A 0.50, B 0.30, C 0.10, D 0.05, E 0.03, F 0.02
top-k = 3 -> candidates A, B, C
top-p = 0.9 -> A (0.50) + B (0.30) + C (0.10) = 0.90 -> A, B, C
top-p = 0.8 -> A + B = 0.80 -> A, BAfter trimming, probabilities are renormalised and one token is sampled.
What to say in the interview
"Top-k limits candidates to a fixed number, top-p to the smallest group covering probability p. Top-p adjusts to the model's confidence, so I usually tune it, along with temperature, and leave top-k alone."