RAG & Retrieval · Ground answers in evidence
How do you decide when retrieval found nothing useful?
Short answer
Calibrate a score threshold on labelled queries, combine it with reranker scores and coverage checks, and prefer an explicit abstention over an unsupported answer.
Top-k retrieval always returns k results, even when none are relevant. Without a threshold, the model is forced to answer from noise. Deciding when to abstain is a product decision backed by evidence, not a fixed similarity number copied from a blog post.
- Label a set of answerable and unanswerable queries and inspect score distributions.
- Use reranker scores or a small classifier if raw similarity is poorly calibrated.
- Abstain, ask a clarifying question or hand off to search when confidence is low.
- Log abstentions so you can distinguish missing content from a bad threshold.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1How would you trade false abstentions against wrong answers?
- 2What would you show the user instead of an answer?
- 3Why is raw cosine score often a poor confidence measure?
Say it in the interview
I calibrate the threshold on labelled answerable and unanswerable queries rather than guessing a number. Raw similarity is often poorly calibrated, so I combine it with reranker scores or a small classifier. When confidence is low, I prefer an explicit abstention or a clarifying question over a plausible unsupported answer. I log those cases, because a rising abstention rate could mean missing content or a broken index, not just a strict threshold.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live RAG & Retrieval mock interview and get scored feedback on depth, clarity and confidence.