RAG & Retrieval · Ground answers in evidence
How do you evaluate a RAG system?
Short answer
Evaluate ingestion, retrieval, context and generation separately, then end to end. Use labelled question-passage pairs for retrieval and evidence-based grading for faithfulness.
An end-to-end score cannot tell you which stage to fix. A useful RAG evaluation reports at least four things: did ingestion preserve the content, did retrieval find the right passage, was the packed context relevant, and was the answer faithful and useful.
- Retrieval: recall@k, context precision and whether the gold passage appears.
- Generation: faithfulness to context, answer relevance, citation accuracy.
- Ingestion: parsing fidelity on a sample of real documents.
- End to end: task success, abstention quality, latency and cost.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1How would you build the gold passage labels?
- 2Which metric would you gate a release on?
- 3How do you evaluate the ingestion stage cheaply?
Say it in the interview
I evaluate each stage, because end-to-end scores do not tell me what to fix. For retrieval I check whether the gold passage is in the candidates and how precise the context is. For generation I check faithfulness, relevance and citation accuracy. I also test unanswerable questions, since correct abstention is part of quality. Then I track task success, latency and cost end to end.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live RAG & Retrieval mock interview and get scored feedback on depth, clarity and confidence.