Why can the same sentence need more tokens in Hindi than in English?
Tokenizers are trained mostly on the languages that dominate their data, so common English words become single tokens while Hindi words in Devanagari often split into many small pieces. More tokens means higher cost, slower replies and less room in the context window.
Explain it simply
Imagine a bus service built for one city. Routes there are direct. For a city it barely covers, you must change buses many times for the same distance.
A tokenizer's vocabulary is learned from its training text. If English dominates, "water" is one token. A Hindi word like "पानी" may be broken into several pieces, sometimes down to bytes, because those character sequences were not frequent enough to earn their own entries.
Consequences are practical. The same message costs more tokens, takes longer to generate, and fills the context window faster. For an app serving Indian users, measure token counts on real Hindi, Hinglish and regional text before estimating cost or limits.
Example
English: "Where is the railway station?" -> few tokens
Hindi: "रेलवे स्टेशन कहाँ है?" -> usually more tokens for the same meaningExact counts depend on the tokenizer, so always count with the one your model uses.
What to say in the interview
"Token efficiency depends on how well the tokenizer's training data covered the language. Underrepresented scripts split into more tokens, which raises cost and latency and shrinks effective context. I would measure on real data and pick a model or tokenizer with good coverage for the target languages."