AI Engineering · LLM applications that ship
What are tokens and context windows?
Short answer
Tokens are the subword units a model reads and writes. The context window is the maximum number of tokens it can attend to in one call, and input plus output share that budget.
Tokenisers split text into pieces that are usually smaller than a word. English averages roughly four characters per token, but code, numbers, non-Latin scripts and rare words tokenise less efficiently. Pricing, rate limits and the context window are all measured in tokens, not words.
- The window covers the whole request: system prompt, conversation history, retrieved documents and the response.
- Longer inputs raise cost and usually raise time to first token.
- A full window is not a target. Irrelevant context can reduce answer quality and makes failures harder to debug.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1What happens when a conversation exceeds the window?
- 2How would you estimate cost per request before shipping?
- 3Why can a larger context window still perform worse than retrieval?
Say it in the interview
Tokens are the chunks of text the model actually processes, roughly four characters each in English. The context window is the total budget for one call, and the prompt plus the completion both count against it. That budget drives cost, latency and rate limits, so I treat it as a resource to allocate, not a limit to fill. For long histories I summarise or retrieve instead of pasting everything.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live AI Engineering mock interview and get scored feedback on depth, clarity and confidence.