Interview Q&A / AI & Generative AI

1. When would you use prompting, RAG or fine-tuning to adapt an LLM?

   Start with prompting because it is cheapest and fastest to change. Add RAG when the model needs knowledge it does not have, such as private or changing data. Use fine-tuning when you need consistent behaviour, style or format that prompting cannot reliably achieve, not mainly to add facts.

   AdvancedIn-depth4 min read
2. What is self-consistency in prompting?

   Self-consistency samples several independent reasoning answers to the same question at a non-zero temperature and picks the most common final answer. It improves accuracy on reasoning tasks at the price of multiplying token cost and latency.

   AdvancedQuick answer1 min read
3. What is the difference between a bi-encoder and a cross-encoder?

   A bi-encoder embeds the query and each document separately, so document vectors can be precomputed and searched fast. A cross-encoder reads query and document together for a more accurate score, but must run for every pair, so it is used on a short candidate list.

   AdvancedQuick answer1 min read