VocalHyre
Sign up

AI Engineering · LLM applications that ship

Fine-tuning vs prompting vs RAG: how do you decide?

Hardfine-tuningragpromptingUpdated Oct 2026
Quiz this track

Short answer

Use prompting for behaviour and format, RAG for knowledge that changes or must be cited, and fine-tuning for consistent style or task patterns that prompting cannot achieve cheaply. They combine.

These solve different problems. Prompting changes instructions at request time. RAG adds external knowledge at request time. Fine-tuning changes the model's weights so a behaviour becomes more reliable or cheaper to elicit. Choosing the wrong one adds cost without fixing the failure.

  • Knowledge that changes, needs citations or varies by customer: RAG.
  • Tone, output format and narrow task behaviour: prompting first, fine-tuning if it is still inconsistent at scale.
  • Latency or cost reduction by replacing a long prompt: fine-tuning can help once you have enough examples.
  • Fine-tuning is not a reliable way to add fresh facts; it can memorise and still hallucinate.
Start with the cheapest intervention and measure. Fine-tuning adds training, evaluation, versioning and deployment overhead that a better prompt or retrieval fix may avoid.

Follow-ups you should expect

Interviewers use these to check whether the first answer was understanding or memorisation.

  1. 1How would you know fine-tuning actually helped?
  2. 2Why is fine-tuning a poor fit for frequently updated policies?
  3. 3What data would you need before considering fine-tuning?

Say it in the interview

I match the tool to the failure. If the model lacks current or private knowledge, that is retrieval. If it ignores format or tone instructions, I improve the prompt first, and only consider fine-tuning if the behaviour stays inconsistent at scale. Fine-tuning can also cut cost by replacing a long prompt, but it needs clean examples and its own evals. I would not use it to inject facts that change often, because it memorises unreliably.

Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.

Related questions

MediumHow does a RAG pipeline work end to end?MediumHow do you get reliable structured output from an LLM?MediumHow do you choose a model for a feature?

Reading is step one

Now say it out loud.

Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live AI Engineering mock interview and get scored feedback on depth, clarity and confidence.

Start a free AI Engineering interviewBrowse all questions