RAG & Retrieval · Ground answers in evidence
RAG vs long context windows: what actually differs?
Short answer
Long context sends more text per call; RAG selects less text per call. Long context is simpler but costs more and can dilute attention. RAG scales across large corpora and supports permissions.
Large context windows make it tempting to paste everything. For a small, stable document set that can work well. For a large or permissioned corpus, it is expensive, slow and often less accurate because relevant evidence is buried among irrelevant text.
- Cost scales with input tokens on every request, not once at ingest.
- Latency grows with context length, especially time to first token.
- Permissions cannot be enforced by asking the model to ignore text it should not see.
- Long context is a good fallback when retrieval misses and the document set is small.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1When would you choose long context over retrieval?
- 2Why is permission enforcement harder with long context?
- 3How would you compare quality between the two approaches?
Say it in the interview
Long context and RAG trade different resources. Long context is simpler, but I pay for every token on every request, latency rises, and the model has to find the signal in a lot of noise. RAG selects a small set of evidence and can enforce permissions at retrieval time. For a handful of stable documents I might just use context. For a large or access-controlled corpus, retrieval is the safer default.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live RAG & Retrieval mock interview and get scored feedback on depth, clarity and confidence.