LLM Evals · Turn quality into a signal
How do you measure groundedness?
Short answer
Decompose the answer into claims and check whether each is supported by the provided evidence. Report unsupported claims and citation accuracy, not just an overall score.
Groundedness asks whether the answer is supported by the context it was given, regardless of whether it happens to be true in the world. The evaluation must have access to the exact evidence supplied to the generator, or it is measuring something else.
- Split the answer into atomic claims.
- For each claim, retrieve the supporting sentence or mark it unsupported.
- Check that citations point to passages that actually support the claim.
- Distinguish unsupported from contradicted: both matter, but they need different fixes.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1How would you handle a true claim that is not in the context?
- 2Why is claim decomposition more useful than one overall score?
- 3How do you measure citation support separately?
Say it in the interview
I decompose the answer into claims and check each against the exact evidence supplied to the generator. A claim that is true but absent from the context is still ungrounded for this purpose. I report unsupported and contradicted claims separately, because they point to different fixes, and I check that citations point to passages that actually support the sentence. The judge must not use outside knowledge.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live LLM Evals mock interview and get scored feedback on depth, clarity and confidence.