RAG & Retrieval · Ground answers in evidence
What chunking strategies exist and when do they work?
Short answer
Fixed-size, recursive, structure-aware and semantic chunking are the main options. Structure-aware splitting usually wins for documents; semantic chunking can help when boundaries are unclear.
Chunking defines the retrieval unit. Fixed windows are simple but cut across ideas. Recursive splitting respects paragraphs and sentences. Structure-aware splitting follows headings and sections. Semantic chunking groups sentences by embedding similarity, which is slower and harder to tune.
- Structure-aware: best default for documentation, policies and manuals.
- Recursive character or token splitting: a solid fallback for messy text.
- Fixed size: predictable and cheap, but weak on long arguments.
- Semantic chunking: useful for transcripts or unstructured prose, with higher cost.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1How would you compare two chunkers on retrieval quality?
- 2Why keep a link from chunk to parent document?
- 3What would you do for very short documents?
Say it in the interview
I default to structure-aware chunking because headings and sections carry meaning. For messy text I fall back to recursive splitting on paragraphs and sentences. Fixed windows are predictable but often cut through an argument, and semantic chunking can help on transcripts but costs more and is harder to tune. I always keep the parent document and section, so a hit can be expanded or cited accurately.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live RAG & Retrieval mock interview and get scored feedback on depth, clarity and confidence.