AI Engineering · LLM applications that ship
How do you validate and repair model output?
Short answer
Parse, validate against the schema and business rules, then repair with one bounded retry or a deterministic fallback. Log every invalid output so the failure rate is visible.
Syntax validity is the minimum. Output also has to satisfy business constraints: enum membership, numeric ranges, referential integrity, required citations and safety rules. Validation is the boundary between model output and the rest of the system.
- Validate structure and semantics in one place with a clear error type.
- Repair by sending the validation error back once, or by extracting the valid subset.
- Fall back to a safe response when repair fails; never pass invalid data downstream.
- Track invalid-output rate by prompt version and model.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1When would you repair rather than retry from scratch?
- 2What would you log to diagnose a rising invalid-output rate?
- 3How do you validate citations, not just JSON?
Say it in the interview
I parse and validate against both the schema and the business rules, then either repair once with the error attached or return a safe fallback. Invalid output never reaches downstream systems. I log the failure rate by prompt and model version, because a rising rate usually means a prompt or model change. I avoid repairs that silently change the meaning of a value; an explicit failure is easier to handle than a plausible wrong number.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live AI Engineering mock interview and get scored feedback on depth, clarity and confidence.