LLM applications that ship
AI Engineering Interview Questions
LLM APIs, prompting, structured output, RAG, agents, evals, guardrails and production concerns. The engineering questions behind applied AI roles.
40 questions5 easy29 medium6 hard0/40 learned
01How do LLMs generate text?Easyllm basics · transformers · inference02What are tokens and context windows?Easytokens · context window · cost03How do temperature, top-p and sampling affect output?Mediumsampling · temperature · determinism04System vs user prompts: what belongs where?Easyprompting · security · instructions05How do you get reliable structured output from an LLM?Mediumstructured output · json · validation06What is prompt injection and how do you defend against it?Hardsecurity · prompt injection · untrusted input07How do you manage context in a long-running application?Mediumcontext · memory · summarisation08Why do LLMs hallucinate?Mediumhallucination · reliability · grounding09How does a RAG pipeline work end to end?Mediumrag · retrieval · architecture10How do you choose a chunking strategy?Mediumchunking · rag · retrieval11How do embeddings and similarity search work?Mediumembeddings · semantic search · vectors12Why combine keyword and vector search?Mediumhybrid search · bm25 · retrieval13What does a reranker add to retrieval?Mediumreranking · precision · retrieval14Where does RAG fail in production?Hardrag · failure modes · debugging15Why are evaluations mandatory for LLM features?Easyevals · quality · regression16Offline vs online evaluation: what does each catch?Mediumevals · online · offline17How do you build a golden evaluation set?Mediumevals · dataset · quality18How do you use an LLM as a judge safely?Hardevals · llm as judge · rubrics19How do you regression-test prompt changes?Mediumprompts · regression · evals20How do you measure RAG quality?Mediumrag · evals · retrieval metrics21Which metrics matter for LLM applications?Mediumevals · metrics · quality22How does tool/function calling work?Mediumtools · function calling · agents23What is an agent loop and when is it worth it?Mediumagents · orchestration · control flow24How do you design tool schemas for an agent?Mediumtools · schemas · agents25How do agents fail on multi-step tasks?Hardagents · failure modes · reliability26When should a human stay in the loop?Mediumhuman in the loop · risk · approval27How should an agent store and use memory?Mediumagents · memory · context28How do you design an LLM latency budget?Mediumlatency · performance · streaming29How do you control LLM cost in production?Mediumcost · tokens · routing30When and how should you cache LLM calls?Mediumcaching · cost · latency31How do you handle rate limits, retries and timeouts?Mediumreliability · retries · rate limits32How do you stream LLM output to users?Easystreaming · ux · latency33What should you trace in an LLM application?Mediumobservability · tracing · debugging34How do you version and ship prompts?Mediumprompts · versioning · deployment35How do you choose a model for a feature?Mediummodels · trade-offs · evaluation36What guardrails belong around an LLM feature?Mediumguardrails · safety · policy37How do you handle PII and sensitive data with LLM APIs?Hardprivacy · pii · compliance38How do you validate and repair model output?Mediumvalidation · structured output · reliability39How should an LLM feature degrade when the model fails?Mediumreliability · fallbacks · graceful degradation40Fine-tuning vs prompting vs RAG: how do you decide?Hardfine-tuning · rag · prompting
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live AI Engineering mock interview and get scored feedback on depth, clarity and confidence.