Interview Q&A / AI & Generative AI
AI & Generative AI interview questions
LLMs, tokens, embeddings, RAG and prompt engineering, the questions every AI role now asks.
Prefer rapid revision? Practise AI & Generative AI short interview answers →
- How would you evaluate an LLM application before launching it?
Build a test set of realistic inputs with expected outcomes, define metrics tied to the real goal, run automated checks plus human review on a sample, include edge cases and adversarial inputs, set pass thresholds, and keep the set to catch regressions later.
BeginnerQuick answer1 min read - What is a golden dataset in LLM evaluation?
A golden dataset is a curated set of inputs paired with verified expected answers or grading criteria, reviewed by people. It is the fixed benchmark you re-run after every change to see whether quality improved or regressed on your real use case.
BeginnerQuick answer1 min read - What is regression testing for prompts and models?
It means re-running a fixed evaluation set whenever you change a prompt, model, retrieval setting or tool, and comparing scores with the previous version. It catches cases where a change fixes one problem but quietly breaks others.
BeginnerQuick answer1 min read - What is the difference between offline and online evaluation of an LLM feature?
Offline evaluation tests a model or prompt on a fixed dataset before release, which is repeatable and safe. Online evaluation measures real behaviour after release, through user feedback, behavioural signals and A/B tests, and catches problems the dataset missed.
BeginnerQuick answer1 min read