AI Engineering · LLM applications that ship
How do temperature, top-p and sampling affect output?
Short answer
Temperature rescales the next-token distribution; top-p and top-k truncate it. Lower temperature is more repeatable, higher is more diverse, but none of them make output factually correct.
The model produces a score for every possible next token. Temperature divides those scores before the softmax: near zero it concentrates probability on the top choice, while higher values flatten the distribution. Top-p keeps the smallest set of tokens whose combined probability reaches p, and top-k keeps the k highest-scoring tokens.
- Use low temperature for extraction, classification and structured output.
- Use moderate temperature for drafting and brainstorming where variation helps.
- Do not rely on temperature zero for exact determinism: batching, hardware and model updates can still change results.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1Would you use the same settings for a summariser and a brainstormer?
- 2How would you test that a temperature change did not break quality?
- 3What does a seed guarantee, and what does it not?
Say it in the interview
Temperature controls how sharp or flat the next-token distribution is, and top-p and top-k cut off the tail. For extraction or JSON I keep it low so the output is stable. For ideation I raise it a little to get variety. I would not claim temperature zero is fully deterministic, and I would never use sampling settings as a substitute for grounding when the model needs facts.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live AI Engineering mock interview and get scored feedback on depth, clarity and confidence.