Why do we split documents into chunks before indexing them for RAG?
Embedding a whole long document blurs many topics into one vector, and passing it all to the model wastes context. Chunks are small enough that each vector represents one idea, so retrieval can return only the relevant passages.
Explain it simply
Suppose your college handbook is 200 pages. A student asks, "What is the attendance rule for exams?" You would not hand over all 200 pages. You would point to the one page about attendance.
Chunking does that for a RAG system. One embedding for a whole handbook mixes hostel rules, fees, exams and holidays into a single average, and a query about attendance matches it weakly. Splitting into paragraph-sized pieces gives each vector a focused meaning, so the attendance chunk scores high.
It also saves tokens. The model receives a few relevant chunks, not the whole document, which lowers cost, speeds up answers and reduces the chance that irrelevant text distracts it.
Example
Handbook (200 pages)
-> chunk 1: "Attendance: minimum 75% to sit semester exams..."
-> chunk 2: "Hostel: gates close at 10 pm..."
-> chunk 3: "Fees: instalment due dates..."
Query "exam attendance rule" retrieves chunk 1 only.What to say in the interview
"Chunking makes each embedding represent one focused idea and lets retrieval return only relevant passages. That improves precision and keeps the prompt small, which cuts cost and latency."