AI Engineering · LLM applications that ship
What does a reranker add to retrieval?
Short answer
A reranker scores each query-document pair jointly and reorders the top candidates. It improves precision after fast recall, at the cost of extra latency and compute.
Vector search is built for speed over millions of chunks, so it optimises recall rather than fine-grained relevance. A reranker takes the top 20–100 candidates and reads the query and passage together, producing a more accurate score. Only the best few passages then reach the prompt.
- Use recall-oriented retrieval first, then rerank a bounded candidate set.
- Budget the reranker in the latency estimate; cross-encoders are much slower than vector lookup.
- Measure context precision before and after to prove the reranker earns its cost.
- Keep a fallback path when the reranker times out.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1How many candidates would you rerank, and why?
- 2When would you skip reranking entirely?
- 3How does reranking change the prompt construction step?
Say it in the interview
Retrieval is optimised for recall, so the first pass may include passages that are only loosely related. A reranker reads the query and each candidate together and gives a much better relevance score. I rerank a bounded set, maybe the top fifty, and send only the best handful to the model. It costs latency, so I measure context precision before and after to confirm it is worth keeping.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live AI Engineering mock interview and get scored feedback on depth, clarity and confidence.