AI Engineering · LLM applications that ship
How do LLMs generate text?
Short answer
An LLM predicts the next token from the preceding tokens, one step at a time. It is a statistical model over token sequences, not a database or a reasoning engine.
A large language model is a transformer trained to model the probability of the next token given everything before it. Generation is a loop: encode the context, score every token in the vocabulary, pick one, append it, and repeat. There is no separate lookup of facts and no persistent state between calls.
- Pre-training teaches general next-token prediction over a very large corpus.
- Instruction tuning and preference training shape the model to follow requests and preferred answer styles.
- At inference time the model only sees the tokens you send plus what it has already generated.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1Why can the same prompt produce different answers?
- 2What does the model actually see on a single call?
- 3Where does its knowledge come from, and where does it stop?
Say it in the interview
An LLM is a next-token predictor. It reads the context I give it, scores what token is most plausible next, emits one, then repeats with that token included. Training on huge text corpora gives it language and general knowledge, and instruction tuning shapes how it responds. It has no memory between calls, so anything I need it to know has to be in the prompt or retrieved into it.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live AI Engineering mock interview and get scored feedback on depth, clarity and confidence.