AI Engineering · LLM applications that ship
How do you choose a chunking strategy?
Short answer
Chunk by document structure first, then by size with overlap. Preserve headings and metadata so each chunk is self-contained enough to answer a question without its neighbours.
Chunking decides the unit of retrieval. Too large and the embedding mixes several ideas, so similarity gets noisy and the prompt fills with irrelevant text. Too small and the chunk loses the context needed to interpret it. Structure-aware splitting usually beats fixed character counts.
- Split on headings, sections, then paragraphs before falling back to fixed token windows.
- Use modest overlap (roughly 10–20 percent) to keep sentences that straddle a boundary.
- Prepend the document title and section path to each chunk before embedding.
- Tune against a retrieval eval set rather than a single anecdotal query.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1How would you evaluate two chunking strategies fairly?
- 2What would you do for a document with very long tables?
- 3Why include the section heading inside the chunk text?
Say it in the interview
I chunk by structure first, so headings and sections stay meaningful, then apply a size limit with a little overlap. Each chunk carries its source and section path so it can be understood on its own and cited. I do not pick a size by intuition; I build a small retrieval eval set and compare strategies on whether the right passage is retrieved. Tables and code need their own handling because naive splitting destroys them.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live AI Engineering mock interview and get scored feedback on depth, clarity and confidence.