AI Engineering · LLM applications that ship
System vs user prompts: what belongs where?
Short answer
The system prompt holds durable role, policy and output contracts. User and tool content carries the task and untrusted data. Never put untrusted text where the model treats it as instructions.
System-level instructions are the closest thing to configuration: who the assistant is, what it must never do, tone, refusal policy and output format. User messages describe the immediate request. Retrieved documents, web pages and tool results are data, even when they contain sentences that look like commands.
- Keep secrets, internal URLs and privileged rules out of any prompt the user can influence.
- Put the output schema in the system prompt and repeat the task-specific fields in the user message.
- Treat instructions found inside retrieved content as untrusted content, not as updates to policy.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1How would you structure a prompt that summarises untrusted emails?
- 2What would you do if a user asks the model to reveal its system prompt?
- 3Where do tool descriptions belong, and why?
Say it in the interview
I use the system prompt for things that should stay constant: role, tone, safety rules and the output contract. The user message carries the request and any untrusted content. That separation matters because text inside a retrieved page or an email is data, not a new set of instructions. I still enforce the important rules in code, because a prompt alone is not a security boundary.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live AI Engineering mock interview and get scored feedback on depth, clarity and confidence.