LLM Evals · Turn quality into a signal
When is exact match the right metric?
Short answer
When there is one correct normalised answer: labels, IDs, dates, structured fields or code output. It is cheap and unambiguous, but brittle to formatting and paraphrase.
Exact match is the strongest signal when the space of valid answers is genuinely small. A sentiment label, a country code or a computed number has one right value. It is the wrong metric for free-form explanations, where many valid phrasings exist.
- Normalise before comparing: whitespace, case, number formatting, units.
- Define acceptable equivalents in advance.
- Use exact match for extraction fields and classification labels.
- Combine with fuzzy or semantic checks for free text.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1How would you normalise dates before exact matching?
- 2When would exact match be too strict even for extraction?
- 3What would you use for a multi-label classification task?
Say it in the interview
Exact match is right when there is one correct answer, such as a label, an ID, a date or a computed number. It is cheap and unambiguous, but I have to normalise formatting and define equivalent forms first. For free-form answers it is too brittle, because many phrasings are valid. When exact match drops to zero, I check the normaliser and parser before blaming the model.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live LLM Evals mock interview and get scored feedback on depth, clarity and confidence.