Why do transformers need positional encodings?
Attention treats its input as an unordered set, so without position information "dog bites man" and "man bites dog" look alike. Positional encodings add order information to the token vectors so the model knows where each token sits.
Explain it simply
Imagine tearing a sentence into word cards and dropping them in a bag. Self-attention alone only sees the bag. It knows which words are present and how they relate by content, but not their order.
Order matters: "Priya called Rahul" is different from "Rahul called Priya". So the model adds a position signal to each token. Early transformers added fixed sine and cosine patterns; many models learn position embeddings or use schemes that encode relative distance between tokens inside the attention calculation.
Position methods also affect how well a model handles text longer than it saw during training, which is one reason context extension is a research topic.
Example
token embedding("Rahul") + position signal(1) -> input for position 1
token embedding("called") + position signal(2) -> input for position 2Same word at a different position gets a different input vector.
What to say in the interview
"Self-attention is permutation-invariant, so we inject order through absolute or relative positional encodings. Without them the model could not tell word order. The chosen scheme also influences how well it generalises to longer sequences."