Prompt Caching Savings: How Much It Cuts Your AI API Bill

Prompt caching is the highest-return, lowest-effort cost lever for most LLM apps. If a big chunk of every request is identical — your system prompt, few-shot examples, tool schemas, or a fixed document — the provider can cache that prefix and bill the repeated tokens at a steep discount instead of full input price. The catch: it only helps when there’s a long, stable prefix that’s reused, and the savings depend on the cache-read rate, which varies by model.

How much does it actually save?

Cache reads only apply to the repeated prefix, but on apps with a large fixed prompt that’s most of the input. Here’s the live cache-read vs standard input pricing for every model that publishes a cached rate:

Read it as: for the cached portion of your prompt, you pay the “cached input” column instead of the “standard input” column. On a 10K-token system prompt reused across thousands of calls, that discount compounds fast.

When prompt caching pays off (and when it doesn’t)

Great fits:

• A large system prompt sent on every request.
• Few-shot examples or a style guide prepended to each call.
• Tool / function definitions included on every agent step.
• A fixed document or schema in a RAG or analysis app.

Poor fits:

• Every request is unique with no shared prefix (nothing to cache).
• The prefix changes each call (cache never hits).
• Very short prompts (below the provider’s minimum cacheable size).

How it works per provider

The mechanics differ — some automatic, some explicit:

• OpenAI — automatic for prompts above a minimum length; the longest matching prefix is cached and discounted with no code change. Keep the stable content at the start of the prompt.
• Anthropic (Claude) — explicit: mark cacheable blocks with cache_control. Cache writes cost a bit more than normal input, and reads are heavily discounted, so it pays off when a block is reused enough to amortize the write.
• Google (Gemini) — supports caching (implicit on some models, explicit context caching on others); discounts apply to the cached context.
• DeepSeek — automatic context caching with a low cache-hit rate on the repeated prefix.

Code: caching a stable prefix

import anthropic
client = anthropic.Anthropic()

BIG_SYSTEM = open("system_prompt.txt").read()  # long, stable

resp = client.messages.create(
  model="claude-sonnet-4-6",
  max_tokens=512,
  system=[
      {
          "type": "text",
          "text": BIG_SYSTEM,
          "cache_control": {"type": "ephemeral"},  # cache this block
      }
  ],
  messages=[{"role": "user", "content": "Answer the question above for: ..."}],
)
# First call writes the cache; subsequent calls within the TTL read it at a discount.
print(resp.usage)  # inspect cache_creation_input_tokens / cache_read_input_tokens

from openai import OpenAI
client = OpenAI()

# No special flag — OpenAI caches the longest repeated prefix automatically.
# Just keep the big stable prompt at the START so it forms a cacheable prefix.
resp = client.chat.completions.create(
  model="gpt-5-4-mini",
  messages=[
      {"role": "system", "content": BIG_STABLE_SYSTEM_PROMPT},  # cached
      {"role": "user", "content": variable_user_input},          # not cached
  ],
)
print(resp.usage)  # prompt_tokens_details.cached_tokens shows the cache hit

Watch the gotchas

• Cache writes can cost more (Anthropic): the first call that populates the cache is billed above standard input, so caching only pays off when the prefix is reused enough to amortize it.
• TTL / eviction: caches expire (often minutes). Low-traffic prefixes may expire between hits and never pay off.
• Minimum size: very short prefixes aren’t cacheable.
• Prefix discipline: any change near the top of the prompt invalidates the cache below it.

Put numbers on your case

Estimate the saving for your real prompt: take your cached-prefix token count × (standard − cached) rate × requests/month. Or price your whole workload model-by-model in the cost calculator, and see current cached rates on the pricing comparisons.

What to do next

1. Find your stable prefix (system prompt, few-shot, tools, fixed context) and confirm it’s reused across requests.
2. Put it first in the prompt so it forms a cacheable prefix.
3. Enable caching — automatic on OpenAI/DeepSeek; add cache_control on Anthropic.
4. Verify hits in the response usage fields, then size the saving against the table above.