RAG vs fine-tuning vs prompt engineering: which to use in 2026

For most business AI features in 2026 the order is simple: get the prompt right first, add retrieval-augmented generation (RAG) when the model needs facts it does not have, and fine-tune only when a narrow, high-volume task already works with prompting but costs too much or runs too slowly. The landscape also changed this year. OpenAI is winding down its self-serve fine-tuning platform, so fine-tuning is now a choice you make on fewer platforms. This guide shows how to diagnose which technique a problem needs, what each costs at current published prices, and two worked examples with the arithmetic.

Prices checked on the providers' pricing pages in August 2026. The cost calculator below was run with Node.js 24.19.0 and has no dependencies.

Three techniques, three different problems

The three are often presented as alternatives. They are not. Each changes a different thing:

• Prompt engineering changes the instructions: what to do, in what format, with which examples. It costs nothing to change and works with every model.
• RAG changes the information the model sees at request time: the policy paragraph, the product record, the contract clause. It is the right tool when the model lacks facts, especially private or changing ones.
• Fine-tuning changes the model's weights by training on examples. It is good at teaching behaviour, such as a labelling scheme, a house style or a fixed output shape, so you can use shorter prompts or a smaller model. It is poor at teaching facts, because facts change and retraining is slow.

The useful question is therefore not "which is best?" but "why are the failing cases failing?". That needs a set of test cases with expected outputs. Without one, any technique choice is a guess.

What changed in 2026: where you can still fine-tune

Fine-tuning availability matters to the decision, because you cannot choose an option your provider no longer sells.

Sources: OpenAI deprecations, OpenAI pricing, Vertex AI pricing, Vertex AI supervised tuning and Anthropic's Claude 3 Haiku fine-tuning article. OpenAI's own optimisation guide now points developers to prompting, few-shot examples and evals instead.

Open-weight models remain an option if you need full control or data residency: you can fine-tune Llama, Gemma or Qwen yourself or on a managed service. Vertex AI lists supervised tuning for Gemma 3 27B at $6.83, Llama 3.3 70B at $6.72 and Qwen 3 32B at $6.57 per 1M training tokens. You then pay to host the model, which for most small teams outweighs any saving.

Cost, latency and maintenance compared

A few points the table cannot show:

• Prompt caching changed the maths. The classic argument for fine-tuning was "long prompts cost too much at volume". Cached prefixes are now billed at a fraction of the input price: on Claude Sonnet 4.6 a cache hit costs $0.30 per 1M tokens against $3.00 uncached, and on Gemini 3.1 Flash-Lite $0.025 against $0.25. A long, stable instruction block with dozens of examples is far cheaper than it was.
• Context windows grew. Claude 4.6 and later models (except Haiku 5.5) bill a 1M-token context at standard rates. That makes "put the whole handbook in the prompt" technically possible, but not cheap or fast, as the second example shows.
• Fine-tuned models age. Every new base model means a new training run and a new evaluation. Your fine-tune of last year's model may now be slower and worse than this year's base model with a good prompt.
• RAG adds moving parts. An index, an embedding model, chunking rules and a search step all need monitoring. The pay-off is that updating knowledge is just re-indexing a document.

A cost calculator you can adapt

The two examples below come from this script. Prices are USD per 1M tokens from the pricing pages linked above; change the constants and the volumes to match your workload.

// Monthly cost of three ways to ship the same LLM feature. Prices are USD per 1M tokens,
// copied from the providers' pricing pages; change them when the pages change.
const PRICES = {
  'gemini-3.1-flash-lite': { input: 0.25, cachedInput: 0.025, output: 1.50, tuningPer1M: 3.00, tunedMultiplier: 1.5 },
  'claude-sonnet-4-6':     { input: 3.00, cachedInput: 0.30,  output: 15.00 },
};

const cost = (tokens, pricePer1M) => (tokens / 1e6) * pricePer1M;
const usd = (n) => '$' + n.toFixed(2).padStart(9);

function promptOnly({ model, requests, fixedPromptTokens, variableTokens, outputTokens, cacheHit = 0 }) {
  const p = PRICES[model];
  const fixed = requests * fixedPromptTokens;
  return cost(fixed * (1 - cacheHit), p.input) + cost(fixed * cacheHit, p.cachedInput)
       + cost(requests * variableTokens, p.input) + cost(requests * outputTokens, p.output);
}

function fineTuned({ model, requests, promptTokens, outputTokens, examples, tokensPerExample, epochs }) {
  const p = PRICES[model];
  const training = cost(examples * tokensPerExample * epochs, p.tuningPer1M);
  const monthly = cost(requests * promptTokens, p.input * p.tunedMultiplier) + cost(requests * outputTokens, p.output * p.tunedMultiplier);
  return { training, monthly };
}

console.log('Case 1: classify 500,000 support tickets a month (Gemini 3.1 Flash-Lite)');
const fewShot = { model: 'gemini-3.1-flash-lite', requests: 500_000, fixedPromptTokens: 3_000, variableTokens: 200, outputTokens: 10 };
console.log(`  few-shot prompt, no caching      ${usd(promptOnly(fewShot))}/month`);
console.log(`  few-shot prompt, cached prefix   ${usd(promptOnly({ ...fewShot, cacheHit: 1 }))}/month`);
const ft = fineTuned({ model: 'gemini-3.1-flash-lite', requests: 500_000, promptTokens: 250, outputTokens: 10, examples: 2_000, tokensPerExample: 300, epochs: 3 });
console.log(`  fine-tuned model                 ${usd(ft.monthly)}/month + $${ft.training.toFixed(2)} per training run`);

console.log('\nCase 2: answer 20,000 policy questions a month from 150k tokens of documents (Claude Sonnet 4.6)');
const stuffed = { model: 'claude-sonnet-4-6', requests: 20_000, fixedPromptTokens: 150_000, variableTokens: 100, outputTokens: 400 };
console.log(`  whole corpus in prompt, cached   ${usd(promptOnly({ ...stuffed, cacheHit: 1 }))}/month (best case)`);
console.log(`  whole corpus in prompt, no cache ${usd(promptOnly(stuffed))}/month`);
console.log(`  RAG, 5,000-token context         ${usd(promptOnly({ model: 'claude-sonnet-4-6', requests: 20_000, fixedPromptTokens: 0, variableTokens: 5_000, outputTokens: 400 }))}/month + embedding and search`);

Example 1: classifying 500,000 support tickets a month

Assumptions: each ticket is about 200 tokens, the answer is a label of about 10 tokens, and a good few-shot prompt with instructions and 30 labelled examples is 3,000 tokens. Model: Gemini 3.1 Flash-Lite at $0.25 input, $0.025 cached input and $1.50 output per 1M tokens.

• Few-shot, no caching: 500,000 × 3,200 = 1.6B input tokens × $0.25 = $400.00, plus 5M output tokens × $1.50 = $7.50. Total $407.50 a month.
• Few-shot, cached prefix: 1.5B cached tokens × $0.025 = $37.50, plus 100M uncached tokens × $0.25 = $25.00, plus $7.50 output. Total $70.00 a month (context-cache storage, if you use explicit caching, is billed separately).
• Fine-tuned: the label scheme moves into the weights, so the prompt shrinks to about 250 tokens. Inference at 1.5 times the base rate: 125M × $0.375 = $46.88, plus 5M × $2.25 = $11.25. Total $58.13 a month, plus training at 2,000 examples × 300 tokens × 3 epochs = 1.8M tokens × $3 = $5.40 per run.

Fine-tuning saves about $12 a month against a cached prompt here. That will not pay for the work of collecting 2,000 labelled examples, re-running training when the categories change, and re-evaluating every new model release. Caching the prompt delivers most of the saving with none of the maintenance. Fine-tuning starts to make sense at much larger volumes, when you need to move to a smaller, faster model to meet a latency target, or when no prompt reaches the accuracy you need.

Example 2: answering 20,000 policy questions a month

Assumptions: 150,000 tokens of policies and procedures, questions of about 100 tokens, answers of about 400 tokens. Model: Claude Sonnet 4.6 at $3.00 input, $0.30 cache hits and $15.00 output per 1M tokens.

• Whole corpus in every prompt, always cached: 3B cached tokens × $0.30 = $900.00, plus 2M question tokens × $3.00 = $6.00, plus 8M output tokens × $15.00 = $120.00. Total $1,026 a month, and that is the best case: every cache write costs $3.75 per 1M and a 5-minute cache will expire between quiet-hour questions.
• Whole corpus, no cache hits: $9,126 a month.
• RAG with about 5,000 tokens of retrieved passages and instructions: 100M × $3.00 = $300.00 plus $120.00 output. Total $420 a month, plus embedding and search costs, which are small next to generation at this volume.

RAG wins on cost, and also on latency, since every stuffed request processes 150,000 tokens. It also gives you citations to the exact passage, which matters when the answer is a policy. Fine-tuning is the wrong tool for this job altogether: policies change, and a model trained on last quarter's refund rules will repeat them confidently. Our RAG pipeline tutorial shows the retrieval side step by step.

Decision guide

1. Write 50 to 200 test cases from real inputs, with expected outputs. Include cases the system should refuse.
2. Prompt first. Clear instructions, a few worked examples, a JSON schema for structured output (see getting valid JSON from LLMs), and try a stronger model before anything else.
3. Read the failures. If the model is missing or misstating facts, add retrieval. If it gets the facts but the format or judgement is off, keep working on the prompt and examples.
4. Cache the stable part of the prompt and measure cost per successful task.
5. Consider fine-tuning only if the task is narrow and stable, volume is high, prompting already reaches acceptable accuracy, and you need a smaller model for cost or latency, or prompting has plateaued below your target. Check that your provider still offers it for the model you want.
6. Combine when it fits. A fine-tuned small model can still use RAG for facts. The techniques stack.

When a feature touches customers or money, the expensive part is rarely the model choice; it is designing retrieval, review steps and evaluation that hold up in production. Our AI business process automation team builds these systems on top of your existing software.

Common mistakes

• Fine-tuning to add knowledge. Training on your product catalogue or policies bakes in a snapshot that goes stale, and the model can still blend it with invented details. Use retrieval for facts.
• Comparing against an uncached prompt. Many fine-tuning business cases compare a fine-tuned model with a prompt nobody cached. Compare against the cached version.
• Skipping the evaluation set. Without one, you cannot tell whether RAG or fine-tuning helped, or whether the next model release broke something.
• Stuffing everything into a long context. It works in a demo and becomes the most expensive line in your cloud bill at volume.
• Building on a platform that is closing. If you rely on OpenAI fine-tuned models, plan for 6 January 2027 now: re-test the base model with a good prompt, or move training to a provider that still offers it.
• Treating retrieved text as trusted. Documents and web pages can contain instructions aimed at the model. Keep them in the data part of the prompt and limit what the model can do with them.

When not to use any of them

If the answer is a lookup or a fixed rule, such as an order status, an opening time or "refund if under 30 days", write normal code. A language model adds cost, latency and a chance of error to a problem a database query solves exactly.

Summary

Prompting fixes instructions, RAG fixes missing facts, and fine-tuning fixes cost or speed for narrow, stable, high-volume behaviour. In 2026, prompt caching and large context windows make prompting cheaper than before, and OpenAI's exit from self-serve fine-tuning makes fine-tuning a narrower choice. Start with a prompt and an evaluation set, add retrieval for facts, and fine-tune only when the numbers prove it pays.

Not sure which approach your AI feature needs? Cloudsoft Zone designs and builds AI business process automation, from evaluation sets to RAG assistants, with costs worked out before you commit.