Interview Q&A / AI & Generative AI
AI & Generative AI interview questions
LLMs, tokens, embeddings, RAG and prompt engineering, the questions every AI role now asks.
Prefer rapid revision? Practise AI & Generative AI short interview answers →
- What are hallucinations in LLMs, and how can you reduce them?
A hallucination is when an LLM produces fluent, confident output that is false or not supported by its input, such as an invented fact or citation. You reduce them by grounding the model in real sources, allowing it to say "I don't know", lowering randomness and checking outputs.
BeginnerIn-depth3 min read - What are tokens and the context window in an LLM?
Tokens are the small pieces of text (whole words, parts of words or symbols) that an LLM reads and writes. The context window is the maximum number of tokens the model can consider at once, including both your input and its output.
BeginnerIn-depth3 min read - What is a large language model (LLM), and how does it generate text?
An LLM is a neural network trained on a huge amount of text to predict the next token. It generates text by predicting one token at a time, adding it to the input, and repeating until it stops.
BeginnerIn-depth3 min read - What is prompt engineering, and what makes a good prompt?
Prompt engineering is writing and testing the input to an LLM so it reliably produces the output you need. Good prompts give clear instructions and context, show a few examples when the format matters, and ask for a specific output structure.
BeginnerIn-depth3 min read - Does telling the model to act as an expert actually improve answers?
A role such as "you are a senior data analyst" can shift tone, vocabulary and the level of detail, but it does not add knowledge the model lacks. Clear task instructions, context and constraints usually matter more than the persona line.
BeginnerQuick answer1 min read - How do you decide how many chunks (top-k) to retrieve in RAG?
Too few chunks risk missing the answer. Too many add noise, raise cost and can bury the useful passage. Choose k by measuring recall and answer quality on test questions, and often retrieve many, rerank, and pass only a handful to the model.
BeginnerQuick answer1 min read - How would you evaluate an LLM application before launching it?
Build a test set of realistic inputs with expected outcomes, define metrics tied to the real goal, run automated checks plus human review on a sample, include edge cases and adversarial inputs, set pass thresholds, and keep the set to catch regressions later.
BeginnerQuick answer1 min read - What are parameters in a language model, and does a bigger model always perform better?
Parameters are the learned numbers, mostly weights, that define what the model does. More parameters generally raise capability, but data quality, training method, and task fit matter too, and larger models cost more and respond slower.
BeginnerQuick answer1 min read - What does it mean that an LLM API is stateless?
The model keeps no memory between calls. Each request must contain everything it needs, including earlier conversation turns. Chat apps create the feeling of memory by resending the history, which also means the history uses tokens every time.
BeginnerQuick answer1 min read - What does next-token prediction mean, and why does it produce fluent text?
The model reads the text so far and outputs a probability for every possible next token, picks one, appends it and repeats. Because it was trained on huge amounts of real text, the likely next token is usually grammatical and on topic.
BeginnerQuick answer1 min read - What happens when you set temperature to 0?
The model effectively always picks the most likely next token, so output becomes close to deterministic and repeatable. It suits extraction and classification, but it is not a guarantee of a correct or perfectly identical answer.
BeginnerQuick answer1 min read - What is a golden dataset in LLM evaluation?
A golden dataset is a curated set of inputs paired with verified expected answers or grading criteria, reviewed by people. It is the fixed benchmark you re-run after every change to see whether quality improved or regressed on your real use case.
BeginnerQuick answer1 min read - What is a knowledge cutoff, and how does it affect an LLM's answers?
The cutoff is the point after which the training data contains nothing, so the model does not know later events unless they are given in the prompt or fetched by a tool. It may also answer confidently as if the world stopped at that date.
BeginnerQuick answer1 min read - What is a vector database, and how is it different from a normal database?
A vector database stores embeddings and finds the ones nearest to a query vector quickly, using specialised indexes. A normal database finds rows by exact values or ranges. Vector search finds similar meaning, not equal values.
BeginnerQuick answer1 min read - What is chunk overlap, and why is it used?
Overlap repeats a small slice of text at the end of one chunk at the start of the next. It prevents a sentence or idea cut at a boundary from being split with no chunk containing it whole, at the cost of some duplicated storage.
BeginnerQuick answer1 min read - What is greedy decoding, and what is its weakness?
Greedy decoding always picks the single most probable next token. It is deterministic and fast, but it can produce repetitive or dull text and can miss better full sentences, because a locally best word may lead to a worse overall answer.
BeginnerQuick answer1 min read - What is regression testing for prompts and models?
It means re-running a fixed evaluation set whenever you change a prompt, model, retrieval setting or tool, and comparing scores with the previous version. It catches cases where a change fixes one problem but quietly breaks others.
BeginnerQuick answer1 min read - What is self-attention in a transformer?
Self-attention lets every token in a sequence look at every other token and take a weighted mix of their information, so each word's representation reflects its context. The weights say how relevant each other token is.
BeginnerQuick answer1 min read - What is the difference between a base model and an instruction-tuned model?
A base model is trained only to continue text, so it may ramble or continue your question with more questions. An instruction-tuned model is further trained on examples of instructions and good answers, so it follows requests and replies helpfully.
BeginnerQuick answer1 min read - What is the difference between a system prompt and a user prompt?
The system prompt holds standing instructions from the developer, such as role, tone, rules and format, and applies to the whole conversation. The user prompt is the per-turn request. Models give system instructions higher priority, but that is not a security guarantee.
BeginnerQuick answer1 min read - What is the difference between an LLM and a chatbot?
An LLM is the underlying model that turns input text into output text. A chatbot is a full product around it, with a chat interface, instructions, memory handling, safety checks, and often tools or company data.
BeginnerQuick answer1 min read - What is the difference between offline and online evaluation of an LLM feature?
Offline evaluation tests a model or prompt on a fixed dataset before release, which is repeatable and safe. Online evaluation measures real behaviour after release, through user feedback, behavioural signals and A/B tests, and catches problems the dataset missed.
BeginnerQuick answer1 min read - What is the difference between training and inference in an LLM?
Training adjusts the model's weights by learning from data and costs enormous compute once. Inference uses the finished, frozen weights to answer a prompt and happens on every request. A chat message never changes the weights.
BeginnerQuick answer1 min read - What is the difference between zero-shot and few-shot prompting?
Zero-shot gives only the instruction. Few-shot adds a handful of worked input and output examples in the prompt so the model copies the format and style. Few-shot often improves consistency without any training.
BeginnerQuick answer1 min read - Which sampling settings would you use for JSON extraction versus brainstorming taglines?
For JSON extraction use low temperature, near zero, so output is stable and format errors are rare. For brainstorming use a higher temperature, with perhaps a moderate top-p, so the model offers varied options. Always validate structured output.
BeginnerQuick answer1 min read - Why can an LLM give different answers to the same question?
Most chat settings sample the next token from a probability distribution instead of always taking the top one, so small random choices early on lead to different wording or even different conclusions. Lowering temperature reduces this variation.
BeginnerQuick answer1 min read - Why did transformers replace RNNs for most language tasks?
RNNs read tokens one at a time and squeeze history into a single state, which is slow to train and forgets distant words. Transformers process all tokens in parallel and connect any two directly through attention, so they train faster and use long context better.
BeginnerQuick answer1 min read - Why do LLMs often fail at counting letters in a word?
The model sees tokens, not individual letters, so a word like "strawberry" may arrive as two or three chunks and the model never directly sees its spelling. It must guess letter counts from patterns, which is unreliable. Using code for counting fixes this.
BeginnerQuick answer1 min read - Why do we split documents into chunks before indexing them for RAG?
Embedding a whole long document blurs many topics into one vector, and passing it all to the model wastes context. Chunks are small enough that each vector represents one idea, so retrieval can return only the relevant passages.
BeginnerQuick answer1 min read - Why must you use the same embedding model for documents and queries?
Each embedding model builds its own vector space, with its own dimensions and meaning for every number. Vectors from different models are not comparable, so mixing them gives meaningless similarity scores, or fails if the sizes differ.
BeginnerQuick answer1 min read