RAG & Retrieval · Ground answers in evidence
What are the most common RAG failure modes?
Short answer
Parsing loss, stale or missing content, retrieval misses, noisy context, ignored evidence and unverifiable citations. Each needs a different fix, so trace the stage before changing the prompt.
RAG failures look similar to users but originate in different stages. A confident wrong answer can mean the document was never indexed, the chunk was split badly, the query missed, or the model ignored good evidence. Changing the prompt fixes only the last case.
- Ingestion: parser drops tables, OCR garbage, duplicates, stale versions.
- Retrieval: vocabulary gap, over-filtering, ANN recall loss, missing permissions.
- Packing: too much noise, truncation, lost source labels.
- Generation: unsupported claims, ignored citations, overconfident partial answers.
- Evaluation: no gold data, so regressions are invisible.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1How would you prioritise fixes across these failure modes?
- 2What trace data would you need for a user-reported error?
- 3Why is prompt tuning a poor first response to a bad answer?
Say it in the interview
The common failures are parsing loss, stale content, retrieval misses, noisy context, ignored evidence and bad citations. They look the same in the final answer, so I trace the stage first. If the right chunk was never retrieved, prompt tuning is wasted. I keep chunk IDs and scores on every request, which lets me turn a vague bug report into a specific stage to fix.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live RAG & Retrieval mock interview and get scored feedback on depth, clarity and confidence.