AI Engineering · LLM applications that ship
Which metrics matter for LLM applications?
Short answer
Pick metrics that match the failure you fear: correctness, groundedness, format validity, refusal behaviour, latency and cost. One composite score hides which of these broke.
A metric is only useful if it maps to a decision. For a support assistant, a wrong but fluent answer is worse than a refusal, so groundedness and escalation behaviour matter more than tone. For a classifier, format validity and label accuracy dominate. Measure the failure modes the product actually cares about.
- Task correctness: did the output achieve the user's goal?
- Groundedness: is each claim supported by the provided evidence?
- Format validity: does structured output parse and satisfy the schema?
- Safety: are refusals, toxicity and PII handling within policy?
- Operational: p50 and p95 latency, cost per request, error and retry rate.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1Which metric would you never average away?
- 2How would you weight quality against latency?
- 3What metric would catch a prompt that became too cautious?
Say it in the interview
I choose metrics from the failure modes that would hurt users. For a RAG assistant that is groundedness and correct escalation; for extraction it is schema validity and field accuracy. I also track latency and cost, because a quality win that doubles response time may not be shippable. I keep the components separate rather than hiding everything in one score, so when quality drops I know which dimension moved.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live AI Engineering mock interview and get scored feedback on depth, clarity and confidence.