Interview Q&A / AI & Generative AI

1. What is retrieval-augmented generation (RAG), and when would you use it?

   RAG first retrieves relevant passages from your own data, then gives them to the LLM along with the question so it answers from that material. Use it when answers must come from private, large or frequently changing information the model was not trained on.

   IntermediateIn-depth3 min read
2. How do you decide how many chunks (top-k) to retrieve in RAG?

   Too few chunks risk missing the answer. Too many add noise, raise cost and can bury the useful passage. Choose k by measuring recall and answer quality on test questions, and often retrieve many, rerank, and pass only a handful to the model.

   BeginnerQuick answer1 min read
3. Why do we split documents into chunks before indexing them for RAG?

   Embedding a whole long document blurs many topics into one vector, and passing it all to the model wastes context. Chunks are small enough that each vector represents one idea, so retrieval can return only the relevant passages.

   BeginnerQuick answer1 min read
4. How do you measure the quality of retrieval in a RAG system?

   Use a set of questions with known correct passages and compute recall at k, the share of questions where a correct chunk appears in the top k, and MRR, which rewards ranking the correct chunk higher. Check retrieval separately from the generated answer.

   IntermediateQuick answer1 min read
5. How does metadata filtering improve retrieval?

   Chunks are stored with fields such as source, date, language, product or access level. Filtering on those fields before or during vector search removes irrelevant or forbidden documents, improving precision and enforcing who is allowed to see what.

   IntermediateQuick answer1 min read
6. What is reranking in RAG, and why add it after retrieval?

   Reranking takes the top candidates from fast retrieval, say 50, and rescores each with a more accurate but slower model that reads the query and passage together. You then send only the best few to the LLM, improving relevance.

   IntermediateQuick answer1 min read
7. What is the difference between a bi-encoder and a cross-encoder?

   A bi-encoder embeds the query and each document separately, so document vectors can be precomputed and searched fast. A cross-encoder reads query and document together for a more accurate score, but must run for every pair, so it is used on a short candidate list.

   AdvancedQuick answer1 min read