AI & Generative AI interview questions and quick answers

Exact search compares the query with every vector, which gets too slow for millions of items. Approximate nearest neighbour indexes such as graph or clustering structures check only a small part of the data and usually return the same top results, trading a little recall for large speed gains.

Every token is compared with every other token, so the number of comparisons grows with the square of the sequence length. Doubling the input roughly quadruples the attention work and memory, which makes very long contexts expensive.

A base model is trained only to continue text, so it may ramble or continue your question with more questions. An instruction-tuned model is further trained on examples of instructions and good answers, so it follows requests and replies helpfully.

A bi-encoder embeds the query and each document separately, so document vectors can be precomputed and searched fast. A cross-encoder reads query and document together for a more accurate score, but must run for every pair, so it is used on a short candidate list.

They measure word overlap with a reference text. A correct answer worded differently scores low, while a fluent wrong answer sharing many words can score high. They suit narrow tasks like translation drafts, but not open-ended answer quality.

Byte pair encoding starts from small units like characters or bytes and repeatedly merges the most frequent adjacent pair into a new token, until a target vocabulary size is reached. Common words become single tokens, rare words split into pieces.

"Exact nearest neighbour is linear in collection size. ANN indexes trade a small recall loss for much lower latency. I would use exact search for small corpora and tune ANN parameters against measured recall when it grows."

To find the closest tea stall to you in a big city, you would not measure the distance to every stall.