Interview Q&A / AI & Generative AI
AI & Generative AI interview questions
LLMs, tokens, embeddings, RAG and prompt engineering, the questions every AI role now asks.
Prefer rapid revision? Practise AI & Generative AI short interview answers →
- What are tokens and the context window in an LLM?
Tokens are the small pieces of text (whole words, parts of words or symbols) that an LLM reads and writes. The context window is the maximum number of tokens the model can consider at once, including both your input and its output.
BeginnerIn-depth3 min read - What does next-token prediction mean, and why does it produce fluent text?
The model reads the text so far and outputs a probability for every possible next token, picks one, appends it and repeats. Because it was trained on huge amounts of real text, the likely next token is usually grammatical and on topic.
BeginnerQuick answer1 min read - Why do LLMs often fail at counting letters in a word?
The model sees tokens, not individual letters, so a word like "strawberry" may arrive as two or three chunks and the model never directly sees its spelling. It must guess letter counts from patterns, which is unreliable. Using code for counting fixes this.
BeginnerQuick answer1 min read - How does subword tokenization such as byte pair encoding work?
Byte pair encoding starts from small units like characters or bytes and repeatedly merges the most frequent adjacent pair into a new token, until a target vocabulary size is reached. Common words become single tokens, rare words split into pieces.
IntermediateQuick answer1 min read - What is the difference between token embeddings and sentence embeddings?
A token embedding is the vector for one token that goes into the model's layers. A sentence embedding is a single vector summarising a whole text, produced by an embedding model, and used for search, clustering and similarity.
IntermediateQuick answer1 min read - Why can the same sentence need more tokens in Hindi than in English?
Tokenizers are trained mostly on the languages that dominate their data, so common English words become single tokens while Hindi words in Devanagari often split into many small pieces. More tokens means higher cost, slower replies and less room in the context window.
IntermediateQuick answer1 min read