Interview Q&A / AI & Generative AI
AI & Generative AI interview questions
LLMs, tokens, embeddings, RAG and prompt engineering, the questions every AI role now asks.
Prefer rapid revision? Practise AI & Generative AI short interview answers →
- How do you choose chunk size in a RAG system?
Small chunks give precise matches but may lose surrounding context. Large chunks keep context but dilute the embedding and waste prompt space. Pick a starting size from the document type, then tune it by measuring retrieval quality on real questions.
IntermediateQuick answer1 min read - How do you make a RAG system cite its sources?
Store a source id with every chunk, pass the chunks to the model with labels, and instruct it to cite the labels it used. Then check in code that each cited id was actually retrieved and ideally that the claim is supported by that chunk.
IntermediateQuick answer1 min read - How do you measure the quality of retrieval in a RAG system?
Use a set of questions with known correct passages and compute recall at k, the share of questions where a correct chunk appears in the top k, and MRR, which rewards ranking the correct chunk higher. Check retrieval separately from the generated answer.
IntermediateQuick answer1 min read - How does metadata filtering improve retrieval?
Chunks are stored with fields such as source, date, language, product or access level. Filtering on those fields before or during vector search removes irrelevant or forbidden documents, improving precision and enforcing who is allowed to see what.
IntermediateQuick answer1 min read - How does subword tokenization such as byte pair encoding work?
Byte pair encoding starts from small units like characters or bytes and repeatedly merges the most frequent adjacent pair into a new token, until a target vocabulary size is reached. Common words become single tokens, rare words split into pieces.
IntermediateQuick answer1 min read - How would you get reliably structured JSON output from an LLM?
Describe the exact schema in the prompt with an example, use the provider's structured output or JSON mode if available, set a low temperature, then parse and validate against a schema in code, retrying or repairing when it fails.
IntermediateQuick answer1 min read - In RAG evaluation, what is the difference between faithfulness and answer relevance?
Faithfulness asks whether every claim in the answer is supported by the retrieved context. Answer relevance asks whether the answer actually addresses the user's question. An answer can be faithful but off-topic, or on-topic but unsupported.
IntermediateQuick answer1 min read - Should you change temperature and top-p at the same time?
Usually adjust one and leave the other at its default. Both reshape the randomness of the choice, so changing both makes it hard to tell which caused a difference and can overshoot, giving output that is either too rigid or too erratic.
IntermediateQuick answer1 min read - What are queries, keys and values in attention?
Each token is projected into three vectors. The query asks what this token is looking for, the key advertises what it contains, and the value is the information it passes on. Query-key matches set weights that mix the values.
IntermediateQuick answer1 min read - What do repetition, frequency and presence penalties do during generation?
They lower the chance of tokens that have already appeared. A frequency penalty grows with each repeat, a presence penalty applies once a token has appeared at all. They reduce loops and repeated phrases but can hurt text that needs repetition.
IntermediateQuick answer1 min read - What is a causal mask in a decoder-only model?
A causal mask blocks each token from attending to later tokens, so position n sees only positions up to n. It matches generation, where future tokens do not exist yet, and lets training predict every position in parallel without cheating.
IntermediateQuick answer1 min read - What is chain-of-thought prompting, and when does it help?
Chain-of-thought prompting asks the model to work through intermediate steps before giving the final answer. It tends to help on multi-step maths, logic and planning problems, but adds tokens, and the shown reasoning is not always a faithful account of how the answer was produced.
IntermediateQuick answer1 min read - What is hybrid search, and why combine keyword and vector search?
Hybrid search runs keyword search such as BM25 and vector search together and merges the rankings. Vector search catches meaning and paraphrases, while keywords catch exact terms like product codes, names and rare jargon that embeddings can miss.
IntermediateQuick answer1 min read - What is LLM-as-a-judge, and what biases should you watch for?
LLM-as-a-judge uses a model to grade another model's outputs against a rubric, scaling evaluation beyond human review. Known biases include preferring longer answers, favouring the first option shown, and favouring its own style, so calibrate against human labels.
IntermediateQuick answer1 min read - What is multi-head attention, and why use several heads?
Multi-head attention runs several attention computations in parallel, each with its own learned projections, then combines the results. Different heads can capture different relationships, such as grammar, reference or nearby words, within the same layer.
IntermediateQuick answer1 min read - What is reranking in RAG, and why add it after retrieval?
Reranking takes the top candidates from fast retrieval, say 50, and rescores each with a more accurate but slower model that reads the query and passage together. You then send only the best few to the LLM, improving relevance.
IntermediateQuick answer1 min read - What is the difference between cosine similarity, dot product and Euclidean distance for vectors?
Cosine similarity compares only direction, dot product mixes direction and length, and Euclidean distance measures straight-line gap. For normalised vectors, cosine and dot product give the same ranking, and Euclidean distance ranks the same way too.
IntermediateQuick answer1 min read - What is the difference between encoder-only, decoder-only and encoder-decoder models?
Encoder-only models like BERT read the whole input in both directions and suit classification and embeddings. Decoder-only models generate text left to right and power most chat LLMs. Encoder-decoder models read an input then generate an output, as in translation.
IntermediateQuick answer1 min read - What is the difference between token embeddings and sentence embeddings?
A token embedding is the vector for one token that goes into the model's layers. A sentence embedding is a single vector summarising a whole text, produced by an embedding model, and used for search, clustering and similarity.
IntermediateQuick answer1 min read - What is top-k sampling, and how does it differ from top-p?
Top-k keeps only the k most likely tokens and samples among them. Top-p keeps the smallest set of tokens whose probabilities add up to p. Top-k uses a fixed count, while top-p adapts the candidate list to how confident the model is.
IntermediateQuick answer1 min read - Why are BLEU and ROUGE poor metrics for judging LLM answers?
They measure word overlap with a reference text. A correct answer worded differently scores low, while a fluent wrong answer sharing many words can score high. They suit narrow tasks like translation drafts, but not open-ended answer quality.
IntermediateQuick answer1 min read - Why can the same sentence need more tokens in Hindi than in English?
Tokenizers are trained mostly on the languages that dominate their data, so common English words become single tokens while Hindi words in Devanagari often split into many small pieces. More tokens means higher cost, slower replies and less room in the context window.
IntermediateQuick answer1 min read - Why do transformers need positional encodings?
Attention treats its input as an unordered set, so without position information "dog bites man" and "man bites dog" look alike. Positional encodings add order information to the token vectors so the model knows where each token sits.
IntermediateQuick answer1 min read - Why do vector databases use approximate nearest neighbour search instead of exact search?
Exact search compares the query with every vector, which gets too slow for millions of items. Approximate nearest neighbour indexes such as graph or clustering structures check only a small part of the data and usually return the same top results, trading a little recall for large speed gains.
IntermediateQuick answer1 min read - Why does attention cost grow quickly as the input gets longer?
Every token is compared with every other token, so the number of comparisons grows with the square of the sequence length. Doubling the input roughly quadruples the attention work and memory, which makes very long contexts expensive.
IntermediateQuick answer1 min read