Interview Q&A / AI & Generative AI
AI & Generative AI interview questions
LLMs, tokens, embeddings, RAG and prompt engineering, the questions every AI role now asks.
Prefer rapid revision? Practise AI & Generative AI short interview answers →
- What are tokens and the context window in an LLM?
Tokens are the small pieces of text (whole words, parts of words or symbols) that an LLM reads and writes. The context window is the maximum number of tokens the model can consider at once, including both your input and its output.
BeginnerIn-depth3 min read - What does next-token prediction mean, and why does it produce fluent text?
The model reads the text so far and outputs a probability for every possible next token, picks one, appends it and repeats. Because it was trained on huge amounts of real text, the likely next token is usually grammatical and on topic.
BeginnerQuick answer1 min read - Why do LLMs often fail at counting letters in a word?
The model sees tokens, not individual letters, so a word like "strawberry" may arrive as two or three chunks and the model never directly sees its spelling. It must guess letter counts from patterns, which is unreliable. Using code for counting fixes this.
BeginnerQuick answer1 min read