What is greedy decoding, and what is its weakness?
Greedy decoding always picks the single most probable next token. It is deterministic and fast, but it can produce repetitive or dull text and can miss better full sentences, because a locally best word may lead to a worse overall answer.
Explain it simply
Imagine choosing a route in Mumbai by always taking the road that looks emptiest at the next junction. Each step looks best, but you can still end up stuck in a dead end that a slightly worse first turn would have avoided.
Greedy decoding does this with words. At every step it takes the highest-probability token. That gives stable, repeatable output, which is useful for extraction, classification and code where you want the most expected answer.
The weaknesses show in open-ended writing. Text can loop, repeating the same phrase, and sound flat, since the safest word is chosen every time. Sampling methods such as temperature, top-k and top-p add controlled randomness, and beam search keeps several candidate sequences to find a higher-probability whole answer.
Example
Prompt: "I love"
Probabilities: " it" 0.30, " you" 0.28, " cricket" 0.20, ...
Greedy always picks " it".
Sampling may pick " cricket" about 20% of the time.What to say in the interview
"Greedy decoding takes the argmax token at each step. It is deterministic and good for tasks with one expected answer, but it can be repetitive and misses globally better sequences. For varied text I sample, using temperature and top-p."