Interview Q&A / AI & Generative AI
AI & Generative AI interview questions
LLMs, tokens, embeddings, RAG and prompt engineering, the questions every AI role now asks.
Prefer rapid revision? Practise AI & Generative AI short interview answers →
- What is self-attention in a transformer?
Self-attention lets every token in a sequence look at every other token and take a weighted mix of their information, so each word's representation reflects its context. The weights say how relevant each other token is.
BeginnerQuick answer1 min read - Why did transformers replace RNNs for most language tasks?
RNNs read tokens one at a time and squeeze history into a single state, which is slow to train and forgets distant words. Transformers process all tokens in parallel and connect any two directly through attention, so they train faster and use long context better.
BeginnerQuick answer1 min read