How to cut LLM API costs with model choice, prompt caching and batching

The first invoice from an LLM provider often surprises small teams. A feature that cost a few dollars in testing can cost hundreds once real users arrive, because every request resends the system prompt, the tool definitions and the conversation so far. The good news is that the biggest savings rarely require a different provider. They come from three levers every major API now offers: choosing the right model per task, caching the parts of the prompt that repeat, and moving work that can wait to a batch API.

This guide explains each lever with the current numbers from Anthropic's and OpenAI's documentation (checked on 7 October 2026), shows a worked example of the arithmetic, and ends with a checklist you can apply to an existing integration this week. Prices and limits change often, so treat the figures as a snapshot and re-check the linked pages before you budget.

Start by measuring where the tokens go

You cannot cut what you have not measured. Both APIs return a usage object with every response. On the Claude Messages API it includes input_tokens, output_tokens, cache_creation_input_tokens and cache_read_input_tokens; OpenAI reports reused tokens in usage.input_tokens_details.cached_tokens. Log these per request, tagged with the feature and customer that caused them. After a few days you will usually find that:

• Input tokens dominate, because long system prompts, tool definitions and retrieved documents are resent on every turn.
• A handful of features or customers produce most of the spend.
• Some calls do not need an immediate answer at all: nightly summaries, classification of yesterday's tickets, evaluation runs.

Each of those findings maps to one of the levers below.

Lever 1: use the smallest model that passes your evaluation

Price differences between model tiers are large. On Anthropic's models overview page the current base prices per million tokens (MTok) are:

Fable 5.1 costs ten times as much per token as Haiku 4.5. The practical approach is to build a small evaluation set (50 to 200 real, anonymised inputs with expected outcomes), run it against two or three tiers, and route each task to the cheapest model that passes. Classification, extraction and short rewrites often run well on a smaller model; multi-step agentic coding is where larger models earn their price. Note too that the Haiku 4.5 retirement date on the same page is "not sooner than October 15, 2026", so plan any migration away from it now.

Output tokens cost five times input tokens on every tier listed. Asking for concise answers, setting a sensible max_tokens, and returning structured JSON instead of prose all reduce the most expensive part of the bill.

Lever 2: prompt caching

Prompt caching stores a prefix of your prompt (tools, system prompt, long reference documents, earlier conversation turns) so that later requests starting with the same prefix are billed at a much lower rate.

Claude: explicit or automatic

On the Claude API you enable caching with cache_control. The simplest form is a single top-level field, which places the breakpoint on the last cacheable block and moves it forward as a conversation grows. For finer control you can put up to four explicit breakpoints on individual blocks. The documented multipliers are:

Prompts shorter than the model's minimum are simply processed without caching, with no error. The minimum is 512 tokens for Fable 5.1, Opus 5.5 and Sonnet 5.5, and 4,096 tokens for Haiku 4.5. A cache entry is only available once the first response has begun, so parallel requests fired at the same instant will not hit each other's cache.

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

const response = await client.messages.create({
  model: "claude-sonnet-5-5",
  max_tokens: 1024,
  cache_control: { type: "ephemeral" }, // automatic caching
  system: LONG_POLICY_DOCUMENT,          // identical on every request
  messages: [{ role: "user", content: customerQuestion }],
});

const u = response.usage;
console.log({
  uncached: u.input_tokens,
  cacheWrite: u.cache_creation_input_tokens,
  cacheRead: u.cache_read_input_tokens,
  output: u.output_tokens,
});

The rule that matters most: the cached prefix must be byte-for-byte identical across requests. A timestamp, a request ID or a shuffled tool list at the top of the system prompt silently defeats the cache. Put stable content first and anything that changes per request last.

OpenAI: on by default

OpenAI's documentation states that prompt caching is enabled by default for supported models. For GPT-5.6 and later the minimum is 1,024 input tokens, cached reads cost 0.1x the uncached rate (0.05x on GPT-6.1 Sol), and the only retention setting is 30 minutes via prompt_cache_options.ttl. An optional prompt_cache_key lets you separate accounting by customer. The same prefix-ordering rule applies.

Lever 3: batch anything that can wait

Both providers offer an asynchronous batch API at half price.

On Claude, batch and caching discounts stack, but because batched requests run concurrently and in any order, cache hits are best-effort; Anthropic reports typical hit rates of 30% to 98% depending on traffic. For batches with a shared prefix it suggests the 1-hour cache duration. Requests that expire are not billed.

const batch = await client.messages.batches.create({
  requests: tickets.map((t) => ({
    custom_id: `ticket-${t.id}`,           // 1-64 chars: letters, digits, - and _
    params: {
      model: "claude-haiku-4-5",
      max_tokens: 200,
      system: CLASSIFIER_INSTRUCTIONS,
      messages: [{ role: "user", content: t.body }],
    },
  })),
});
console.log(batch.id, batch.processing_status);

Good candidates are nightly report generation, back-filling tags or summaries on old content, translating a product catalogue, and running your evaluation set after every prompt change.

A worked example

Suppose a support assistant on Claude Sonnet 5.5 handles 2,000 requests a day. Each request sends a 6,000-token system prompt and tool definitions that never change, about 500 tokens of customer-specific input, and receives about 400 output tokens. The figures below are our own arithmetic from the published prices, not a measured result.

• No caching: 13 million input tokens a day at $2/MTok is $26, plus 0.8 million output tokens at $10/MTok is $8, for about $34 a day.
• With caching, assuming nearly every request hits the cache: 12 million cached tokens at $0.20/MTok is $2.40, 1 million uncached input tokens is $2, and output is still $8, for about $12.40 a day plus a small amount for cache writes when the cache expires.
• If the same work could run overnight as a batch: the 50% batch discount applies on top, with cache hits no longer guaranteed.

Output is now the largest line item, which points to the next optimisation: shorter answers.

Guardrails that stop runaway spend

• Set a hard spending limit and alerts in the provider console, and a per-customer daily token budget in your own code.
• Cap agent loops (number of turns and total tokens per task). An agent stuck retrying a failing tool can burn thousands of calls.
• Trim conversation history: summarise older turns instead of resending them forever.
• Retrieve fewer, better chunks. Sending ten documents "just in case" is the most common source of bloated input.
• Cache your own results for identical questions where correctness allows.

Checklist

1. Log usage per request with feature and customer tags.
2. Build an evaluation set and test the next-cheaper model tier.
3. Move stable content to the start of the prompt and enable caching; confirm cache reads in the usage fields.
4. Move non-urgent jobs to the batch API.
5. Set max_tokens deliberately and ask for concise or structured output.
6. Add budgets, loop caps and alerts.
7. Re-check prices and model retirement dates every quarter.

Common mistakes that defeat the savings

• Dynamic content at the top of the prompt. Putting today's date, the user's name or a request ID before the long instructions changes the prefix on every request, so nothing is ever read from cache. Move it to the end.
• Reordering tools. Building the tools array from an unordered map can change its order between requests and invalidate the cache. Sort it.
• Prompts below the cache minimum. A 300-token system prompt is never cached on any current Claude model; caching only starts to matter once the shared prefix passes the model's minimum.
• Changing tool_choice between turns. On Claude, this invalidates cached message blocks.
• Batching interactive work. A batch can take up to 24 hours; do not put anything a user is waiting for into one.
• Comparing list prices only. A cheaper model that needs two retries or longer answers can cost more than a dearer one that gets it right first time. Compare cost per successful task on your evaluation set.

Summary

Model choice, prompt caching and batching are independent levers, and they multiply. Measure first, pick the smallest model that passes your own tests, make the repeated part of every prompt identical so it can be cached, and push anything that can wait into a batch. For most small teams those four steps cut the bill substantially without changing what users see.