AI Engineering · LLM applications that ship
Why do LLMs hallucinate?
Short answer
Models optimise for plausible continuations, not truth. Missing knowledge, ambiguous questions and pressure to answer all produce confident fabrication.
A hallucination is fluent output that is not supported by evidence. The model has no internal truth check; it produces the most likely continuation given its training and the prompt. When the answer is rare, recent or absent from the context, a plausible-sounding guess can be statistically reasonable and still wrong.
- Knowledge gaps: the fact was never in training data or has changed since.
- Prompt pressure: the model is asked for a precise answer and has no way to say it does not know.
- Context problems: retrieval returned irrelevant passages, or the instructions encouraged extrapolation.
- Evaluation gaps: no one checks faithfulness, so the failure ships silently.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1How would you detect hallucination in production?
- 2Why does adding more context sometimes make it worse?
- 3What role should confidence scores play?
Say it in the interview
Hallucination happens because the model is predicting plausible text, not checking facts. If the knowledge is missing or the context is weak, it can still produce a fluent answer that sounds certain. I reduce it by grounding in retrieved evidence, asking for citations, and giving the model an explicit way to say it does not have enough information. Then I measure groundedness in evals so regressions are visible.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live AI Engineering mock interview and get scored feedback on depth, clarity and confidence.