RAG & Retrieval · Ground answers in evidence
What happens when documents and queries drift apart?
Short answer
New vocabulary, products and user phrasing can move queries away from how documents were written, so similarity degrades. Detect it with retrieval metrics and query logs, then reindex or adapt.
Embeddings are only meaningful relative to the data they were built from. If the product launches new terminology or users start asking in a different style, the same index can become less effective without any code change. This is distribution drift, and it is invisible in a static eval set.
- Monitor zero-result and low-score retrieval rates over time.
- Sample real queries and compare them with the queries in your eval set.
- Track vocabulary that appears in queries but not in the index.
- Re-run retrieval evals on fresh labelled queries, not only the original set.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1How would you detect drift before users complain?
- 2What would you do if new product terms are missing from the corpus?
- 3How often would you refresh the eval set?
Say it in the interview
Drift shows up as retrieval that slowly stops finding the right passages even though nothing in the code changed. New product names, new user phrasing or a silent embedding-model update can cause it. I monitor low-score and zero-result queries, compare current queries with my eval set, and refresh labelled examples on a schedule. If the corpus is missing new vocabulary, the fix is content and reindexing, not just a different prompt.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live RAG & Retrieval mock interview and get scored feedback on depth, clarity and confidence.