Interview Q&A / AI & Generative AI

1. What is retrieval-augmented generation (RAG), and when would you use it?

   RAG first retrieves relevant passages from your own data, then gives them to the LLM along with the question so it answers from that material. Use it when answers must come from private, large or frequently changing information the model was not trained on.

   IntermediateIn-depth3 min read
2. When would you use prompting, RAG or fine-tuning to adapt an LLM?

   Start with prompting because it is cheapest and fastest to change. Add RAG when the model needs knowledge it does not have, such as private or changing data. Use fine-tuning when you need consistent behaviour, style or format that prompting cannot reliably achieve, not mainly to add facts.

   AdvancedIn-depth4 min read
3. How do you decide how many chunks (top-k) to retrieve in RAG?

   Too few chunks risk missing the answer. Too many add noise, raise cost and can bury the useful passage. Choose k by measuring recall and answer quality on test questions, and often retrieve many, rerank, and pass only a handful to the model.

   BeginnerQuick answer1 min read
4. What is a vector database, and how is it different from a normal database?

   A vector database stores embeddings and finds the ones nearest to a query vector quickly, using specialised indexes. A normal database finds rows by exact values or ranges. Vector search finds similar meaning, not equal values.

   BeginnerQuick answer1 min read
5. What is chunk overlap, and why is it used?

   Overlap repeats a small slice of text at the end of one chunk at the start of the next. It prevents a sentence or idea cut at a boundary from being split with no chunk containing it whole, at the cost of some duplicated storage.

   BeginnerQuick answer1 min read
6. Why do we split documents into chunks before indexing them for RAG?

   Embedding a whole long document blurs many topics into one vector, and passing it all to the model wastes context. Chunks are small enough that each vector represents one idea, so retrieval can return only the relevant passages.

   BeginnerQuick answer1 min read
7. Why must you use the same embedding model for documents and queries?

   Each embedding model builds its own vector space, with its own dimensions and meaning for every number. Vectors from different models are not comparable, so mixing them gives meaningless similarity scores, or fails if the sizes differ.

   BeginnerQuick answer1 min read
8. How do you choose chunk size in a RAG system?

   Small chunks give precise matches but may lose surrounding context. Large chunks keep context but dilute the embedding and waste prompt space. Pick a starting size from the document type, then tune it by measuring retrieval quality on real questions.

   IntermediateQuick answer1 min read
9. How do you make a RAG system cite its sources?

   Store a source id with every chunk, pass the chunks to the model with labels, and instruct it to cite the labels it used. Then check in code that each cited id was actually retrieved and ideally that the claim is supported by that chunk.

   IntermediateQuick answer1 min read
10. How does metadata filtering improve retrieval?

    Chunks are stored with fields such as source, date, language, product or access level. Filtering on those fields before or during vector search removes irrelevant or forbidden documents, improving precision and enforcing who is allowed to see what.

    IntermediateQuick answer1 min read
11. In RAG evaluation, what is the difference between faithfulness and answer relevance?

    Faithfulness asks whether every claim in the answer is supported by the retrieved context. Answer relevance asks whether the answer actually addresses the user's question. An answer can be faithful but off-topic, or on-topic but unsupported.

    IntermediateQuick answer1 min read
12. What is hybrid search, and why combine keyword and vector search?

    Hybrid search runs keyword search such as BM25 and vector search together and merges the rankings. Vector search catches meaning and paraphrases, while keywords catch exact terms like product codes, names and rare jargon that embeddings can miss.

    IntermediateQuick answer1 min read
13. What is reranking in RAG, and why add it after retrieval?

    Reranking takes the top candidates from fast retrieval, say 50, and rescores each with a more accurate but slower model that reads the query and passage together. You then send only the best few to the LLM, improving relevance.

    IntermediateQuick answer1 min read