RAG vs fine-tuning vs prompt engineering: which to use in 2026
Prompting fixes instructions, RAG fixes missing facts, fine-tuning fixes cost at volume. A decision guide with 2026 fine-tuning availability, prices and worked costs.
Updated 11 min read

In this article
Part 1 of 8
Adding AI to a product
All partsHide
Part 1 of 8
Adding AI to a product
Choosing an approach, getting reliable output, then testing it and keeping its cost under control.
RAG vs fine-tuning vs prompt engineering: which to use in 2026
You are here · 11 min read
RAG vs fine-tuning is rarely the first decision: get the prompt right, add retrieval when the model lacks facts, and fine-tune only a narrow, high-volume task that already works with prompting but costs too much or runs too slowly.
For most business AI features in 2026 the order is simple: get the prompt right first, add retrieval-augmented generation (RAG) when the model needs facts it does not have, and fine-tune only when a narrow, high-volume task already works with prompting but costs too much or runs too slowly. The landscape also changed this year. OpenAI is winding down its self-serve fine-tuning platform, so fine-tuning is now a choice you make on fewer platforms. This guide shows how to diagnose which technique a problem needs, what each costs at current published prices, and two worked examples with the arithmetic.
Prices checked on the providers' pricing pages in August 2026. The cost calculator below was run with Node.js 24.19.0 and has no dependencies.
RAG vs fine-tuning vs prompt engineering: three different problems
The three are often presented as alternatives. They are not. Each changes a different thing:
- Prompt engineering changes the instructions: what to do, in what format, with which examples. It costs nothing to change and works with every model.
- RAG changes the information the model sees at request time: the policy paragraph, the product record, the contract clause. It is the right tool when the model lacks facts, especially private or changing ones.
- Fine-tuning changes the model's weights by training on examples. It is good at teaching behaviour, such as a labelling scheme, a house style or a fixed output shape, so you can use shorter prompts or a smaller model. It is poor at teaching facts, because facts change and retraining is slow.

The useful question is therefore not "which is best?" but "why are the failing cases failing?". That needs a set of test cases with expected outputs. Without one, any technique choice is a guess.
What changed in 2026: where you can still fine-tune
Fine-tuning availability matters to the decision, because you cannot choose an option your provider no longer sells.
| Provider | Self-serve fine-tuning status (August 2026) | Example price |
|---|---|---|
| OpenAI | Winding down. Since 7 May 2026 organisations that never ran fine-tuning cannot start; since 2 July 2026 only organisations that ran inference on a fine-tuned model in the last 60 days can create jobs; existing customers lose job creation on 6 January 2027. Existing fine-tuned models keep serving until their base models are deprecated. | gpt-4.1-mini: $5.00 per 1M training tokens; fine-tuned inference $0.80 input and $3.20 output per 1M, double the base model's $0.40 and $1.60 |
| Google Vertex AI | Supervised fine-tuning available for Gemini 3.5 Flash, Gemini 3.1 Flash-Lite, Gemini 2.5 Pro, 2.5 Flash and 2.5 Flash-Lite, plus open models such as Gemma 3, Llama and Qwen | Gemini 3.1 Flash-Lite: $0.003 per 1,000 training tokens ($3 per 1M); for Gemini 3 and later, tuned endpoints cost 1.5 times the base model's prediction price |
| Anthropic | No fine-tuning endpoint on the Claude API. The one Claude fine-tuning offer, launched in 2024, was for Claude 3 Haiku through Amazon Bedrock | Not applicable for current models |
Sources: OpenAI deprecations, OpenAI pricing, Vertex AI pricing, Vertex AI supervised tuning and Anthropic's Claude 3 Haiku fine-tuning article. OpenAI's own optimisation guide now points developers to prompting, few-shot examples and evals instead.
Open-weight models remain an option if you need full control or data residency: you can fine-tune Llama, Gemma or Qwen yourself or on a managed service. Vertex AI lists supervised tuning for Gemma 3 27B at $6.83, Llama 3.3 70B at $6.72 and Qwen 3 32B at $6.57 per 1M training tokens. You then pay to host the model, which for most small teams outweighs any saving.
Cost, latency and maintenance compared

A few points the table cannot show:
- Prompt caching changed the maths. The classic argument for fine-tuning was "long prompts cost too much at volume". Cached prefixes are now billed at a fraction of the input price: on Claude Sonnet 4.6 a cache hit costs $0.30 per 1M tokens against $3.00 uncached, and on Gemini 3.1 Flash-Lite $0.025 against $0.25. A long, stable instruction block with dozens of examples is far cheaper than it was.
- Context windows grew. Claude 4.6 and later models (except Haiku 5.5) bill a 1M-token context at standard rates. That makes "put the whole handbook in the prompt" technically possible, but not cheap or fast, as the second example shows.
- Fine-tuned models age. Every new base model means a new training run and a new evaluation. Your fine-tune of last year's model may now be slower and worse than this year's base model with a good prompt.
- RAG adds moving parts. An index, an embedding model, chunking rules and a search step all need monitoring. The pay-off is that updating knowledge is just re-indexing a document.
A cost calculator you can adapt
The two examples below come from this script. Prices are USD per 1M tokens from the pricing pages linked above; change the constants and the volumes to match your workload.
// Monthly cost of three ways to ship the same LLM feature. Prices are USD per 1M tokens,
// copied from the providers' pricing pages; change them when the pages change.
const PRICES = {
'gemini-3.1-flash-lite': { input: 0.25, cachedInput: 0.025, output: 1.50, tuningPer1M: 3.00, tunedMultiplier: 1.5 },
'claude-sonnet-4-6': { input: 3.00, cachedInput: 0.30, output: 15.00 },
};
const cost = (tokens, pricePer1M) => (tokens / 1e6) * pricePer1M;
const usd = (n) => '$' + n.toFixed(2).padStart(9);
function promptOnly({ model, requests, fixedPromptTokens, variableTokens, outputTokens, cacheHit = 0 }) {
const p = PRICES[model];
const fixed = requests * fixedPromptTokens;
return cost(fixed * (1 - cacheHit), p.input) + cost(fixed * cacheHit, p.cachedInput)
+ cost(requests * variableTokens, p.input) + cost(requests * outputTokens, p.output);
}
function fineTuned({ model, requests, promptTokens, outputTokens, examples, tokensPerExample, epochs }) {
const p = PRICES[model];
const training = cost(examples * tokensPerExample * epochs, p.tuningPer1M);
const monthly = cost(requests * promptTokens, p.input * p.tunedMultiplier) + cost(requests * outputTokens, p.output * p.tunedMultiplier);
return { training, monthly };
}
console.log('Case 1: classify 500,000 support tickets a month (Gemini 3.1 Flash-Lite)');
const fewShot = { model: 'gemini-3.1-flash-lite', requests: 500_000, fixedPromptTokens: 3_000, variableTokens: 200, outputTokens: 10 };
console.log(` few-shot prompt, no caching ${usd(promptOnly(fewShot))}/month`);
console.log(` few-shot prompt, cached prefix ${usd(promptOnly({ ...fewShot, cacheHit: 1 }))}/month`);
const ft = fineTuned({ model: 'gemini-3.1-flash-lite', requests: 500_000, promptTokens: 250, outputTokens: 10, examples: 2_000, tokensPerExample: 300, epochs: 3 });
console.log(` fine-tuned model ${usd(ft.monthly)}/month + $${ft.training.toFixed(2)} per training run`);
console.log('\nCase 2: answer 20,000 policy questions a month from 150k tokens of documents (Claude Sonnet 4.6)');
const stuffed = { model: 'claude-sonnet-4-6', requests: 20_000, fixedPromptTokens: 150_000, variableTokens: 100, outputTokens: 400 };
console.log(` whole corpus in prompt, cached ${usd(promptOnly({ ...stuffed, cacheHit: 1 }))}/month (best case)`);
console.log(` whole corpus in prompt, no cache ${usd(promptOnly(stuffed))}/month`);
console.log(` RAG, 5,000-token context ${usd(promptOnly({ model: 'claude-sonnet-4-6', requests: 20_000, fixedPromptTokens: 0, variableTokens: 5_000, outputTokens: 400 }))}/month + embedding and search`);Example 1: classifying 500,000 support tickets a month
Assumptions: each ticket is about 200 tokens, the answer is a label of about 10 tokens, and a good few-shot prompt with instructions and 30 labelled examples is 3,000 tokens. Model: Gemini 3.1 Flash-Lite at $0.25 input, $0.025 cached input and $1.50 output per 1M tokens.
- Few-shot, no caching: 500,000 × 3,200 = 1.6B input tokens × $0.25 = $400.00, plus 5M output tokens × $1.50 = $7.50. Total $407.50 a month.
- Few-shot, cached prefix: 1.5B cached tokens × $0.025 = $37.50, plus 100M uncached tokens × $0.25 = $25.00, plus $7.50 output. Total $70.00 a month (context-cache storage, if you use explicit caching, is billed separately).
- Fine-tuned: the label scheme moves into the weights, so the prompt shrinks to about 250 tokens. Inference at 1.5 times the base rate: 125M × $0.375 = $46.88, plus 5M × $2.25 = $11.25. Total $58.13 a month, plus training at 2,000 examples × 300 tokens × 3 epochs = 1.8M tokens × $3 = $5.40 per run.
Fine-tuning saves about $12 a month against a cached prompt here. That will not pay for the work of collecting 2,000 labelled examples, re-running training when the categories change, and re-evaluating every new model release. Caching the prompt delivers most of the saving with none of the maintenance. Fine-tuning starts to make sense at much larger volumes, when you need to move to a smaller, faster model to meet a latency target, or when no prompt reaches the accuracy you need.
Example 2: answering 20,000 policy questions a month
Assumptions: 150,000 tokens of policies and procedures, questions of about 100 tokens, answers of about 400 tokens. Model: Claude Sonnet 4.6 at $3.00 input, $0.30 cache hits and $15.00 output per 1M tokens.
- Whole corpus in every prompt, always cached: 3B cached tokens × $0.30 = $900.00, plus 2M question tokens × $3.00 = $6.00, plus 8M output tokens × $15.00 = $120.00. Total $1,026 a month, and that is the best case: every cache write costs $3.75 per 1M and a 5-minute cache will expire between quiet-hour questions.
- Whole corpus, no cache hits: $9,126 a month.
- RAG with about 5,000 tokens of retrieved passages and instructions: 100M × $3.00 = $300.00 plus $120.00 output. Total $420 a month, plus embedding and search costs, which are small next to generation at this volume.
RAG wins on cost, and also on latency, since every stuffed request processes 150,000 tokens. It also gives you citations to the exact passage, which matters when the answer is a policy. Fine-tuning is the wrong tool for this job altogether: policies change, and a model trained on last quarter's refund rules will repeat them confidently. Our RAG pipeline tutorial shows the retrieval side step by step.
Decision guide
- Write 50 to 200 test cases from real inputs, with expected outputs. Include cases the system should refuse.
- Prompt first. Clear instructions, a few worked examples, a JSON schema for structured output (see getting valid JSON from LLMs), and try a stronger model before anything else.
- Read the failures. If the model is missing or misstating facts, add retrieval. If it gets the facts but the format or judgement is off, keep working on the prompt and examples.
- Cache the stable part of the prompt and measure cost per successful task.
- Consider fine-tuning only if the task is narrow and stable, volume is high, prompting already reaches acceptable accuracy, and you need a smaller model for cost or latency, or prompting has plateaued below your target. Check that your provider still offers it for the model you want.
- Combine when it fits. A fine-tuned small model can still use RAG for facts. The techniques stack.
When a feature touches customers or money, the expensive part is rarely the model choice; it is designing retrieval, review steps and evaluation that hold up in production. Our AI business process automation team builds these systems on top of your existing software.
Common mistakes
- Fine-tuning to add knowledge. Training on your product catalogue or policies bakes in a snapshot that goes stale, and the model can still blend it with invented details. Use retrieval for facts.
- Comparing against an uncached prompt. Many fine-tuning business cases compare a fine-tuned model with a prompt nobody cached. Compare against the cached version.
- Skipping the evaluation set. Without one, you cannot tell whether RAG or fine-tuning helped, or whether the next model release broke something.
- Stuffing everything into a long context. It works in a demo and becomes the most expensive line in your cloud bill at volume.
- Building on a platform that is closing. If you rely on OpenAI fine-tuned models, plan for 6 January 2027 now: re-test the base model with a good prompt, or move training to a provider that still offers it.
- Treating retrieved text as trusted. Documents and web pages can contain instructions aimed at the model. Keep them in the data part of the prompt and limit what the model can do with them.
When not to use any of them
If the answer is a lookup or a fixed rule, such as an order status, an opening time or "refund if under 30 days", write normal code. A language model adds cost, latency and a chance of error to a problem a database query solves exactly.
Summary
Prompting fixes instructions, RAG fixes missing facts, and fine-tuning fixes cost or speed for narrow, stable, high-volume behaviour. In 2026, prompt caching and large context windows make prompting cheaper than before, and OpenAI's exit from self-serve fine-tuning makes fine-tuning a narrower choice. Start with a prompt and an evaluation set, add retrieval for facts, and fine-tune only when the numbers prove it pays.
Not sure which approach your AI feature needs? Cloudsoft Zone designs and builds AI business process automation, from evaluation sets to RAG assistants, with costs worked out before you commit.
Frequently asked questions
Is fine-tuning better than RAG?
They solve different problems. RAG gives a model facts at request time and is the right choice for private or changing information. Fine-tuning teaches behaviour such as a label scheme or output style. Many systems need RAG and good prompts; few need fine-tuning.
Can I still fine-tune OpenAI models?
Only if your organisation already uses fine-tuning. OpenAI closed the platform to new users on 7 May 2026, limited job creation on 2 July 2026 to organisations with recent fine-tuned inference, and will stop new jobs for existing customers on 6 January 2027. Existing fine-tuned models keep serving until their base models are deprecated.
Does Claude support fine-tuning?
Not on the Claude API. Anthropic offered fine-tuning for Claude 3 Haiku through Amazon Bedrock in 2024, an older model. For current Claude models, use prompting, prompt caching and retrieval.
How much does fine-tuning a Gemini model cost?
On Vertex AI, supervised tuning of Gemini 3.1 Flash-Lite costs $0.003 per 1,000 training tokens, where training tokens are dataset tokens times epochs. For Gemini 3 and later, a tuned endpoint charges 1.5 times the base model's prediction price. Check the Vertex AI pricing page before you budget.
When is prompt engineering enough?
More often than people expect. If the model has the facts it needs and your test cases fail on format or judgement, better instructions, examples and a schema usually fix it, and caching keeps long prompts affordable.
Sources
- Deprecations: update to OpenAI's self-serve fine-tuning — OpenAI, 2026-05-07
- API pricing (fine-tuning section) — OpenAI
- Generative AI pricing (model tuning) — Google Cloud
- Tune Gemini models with supervised fine-tuning — Google Cloud
- Pricing (prompt caching, long context) — Anthropic
- Fine-tune Claude 3 Haiku in Amazon Bedrock — Anthropic, 2024-07-10
Part 2 of 8: Adding AI to a product
Next part

Troubleshooting guideAI and Automation8 min read
Get valid JSON from LLMs every time with structured outputs
Stop parsing broken JSON from LLMs. How structured outputs work on the Claude and OpenAI APIs, the schema rules that cause 400s, and how to handle refusals.
Comments
No comments yet. Be the first to join the conversation.
Leave a comment
Every comment is read by a moderator before it appears. Your email address is never published; it is used only to contact you about this comment.
More from Cloudsoft Zone

TutorialDevOps5 min read
Deployment approval gates: when a production release needs a human decision
Use deployment approval gates for concrete production decisions with artifact identity, migration risk, acceptance evidence and a clear recovery plan before credentials are released.
By Nilesh Kushvaha

TutorialDevOps5 min read
CI/CD secrets: protect production credentials without exposing them to builds or logs
Manage CI/CD secrets with separate verification and deployment authority, short-lived identity where supported, narrow provider permissions and explicit exposure recovery.
By Nilesh Kushvaha

TutorialDevOps5 min read
Automated rollback for VPS deployments: release identity, compatibility and recovery gates
Design automated rollback around an exact previous release, schema compatibility and verified user recovery instead of blindly reverting every failed deployment.
By Nilesh Kushvaha

ComparisonDevOps5 min read
Blue-green vs rolling vs recreate deployments: choose a strategy your small team can operate
Compare blue-green vs rolling vs recreate deployments through capacity, mixed-version compatibility, worker behavior and realistic recovery requirements for a small team.
By Nilesh Kushvaha
