What is the difference between token embeddings and sentence embeddings?
A token embedding is the vector for one token that goes into the model's layers. A sentence embedding is a single vector summarising a whole text, produced by an embedding model, and used for search, clustering and similarity.
Explain it simply
Consider a cricket team sheet. Each player's profile card is like a token embedding: one card per name. A one-line summary of the team's playing style is like a sentence embedding: one description of the whole group.
Inside an LLM, each token starts as a lookup-table vector and is then repeatedly updated by attention layers, so deep in the network its vector depends on the full context. These vectors are internal working material.
A sentence or document embedding comes from a model trained for that purpose, usually to place texts with similar meaning close together. It outputs one fixed-size vector for the entire input, no matter how long, which makes it easy to store in a vector database and compare.
For search you need the second kind, not the raw token vectors.
Example
Input: "Refund for a cancelled IRCTC ticket"
Token vectors: [Refund] [for] [a] [cancelled] [IRCTC] [ticket] -> 6 vectors
Sentence vector: one vector for the whole line -> stored in the indexWhat to say in the interview
"Token embeddings are per-token inputs to a model. Sentence or document embeddings come from a model trained so that similar meanings are close, producing one vector per text. Semantic search uses the latter."