Prompt Caching for AI Agents: How to Cut Input Costs

An agent resends almost everything on every turn: the system prompt, every tool definition and the whole conversation so far. On a long task, the same few thousand tokens of instructions get billed dozens of times.

Prompt caching fixes most of that. Both Anthropic and OpenAI let you reuse the processed form of a prompt prefix you sent recently, at a fraction of the normal input price. The catch is that the cache only hits if your prompt is built the right way, and agents are very good at accidentally breaking it.

This post covers how caching works, how to structure an agent so it hits, and how to check that it is working.

How prompt caching works

The idea is the same on both platforms. When a request starts with exactly the same tokens as a recent request, the provider can skip reprocessing that shared beginning and charge you less for it.

Two rules follow from that:

Only the prefix counts. The cache matches from the start of the prompt and stops at the first difference. A changed character near the top makes everything after it a miss.
Matches are exact. Anthropic's documentation says cache hits require 100 percent identical prompt segments. "Almost the same" is a miss.

So the whole game is ordering: put what never changes first, and what changes every turn last.

The order that works

For an agent, that usually means:

Tool definitions. Same on every turn.
System prompt. Same on every turn, with no timestamps, user names or request IDs in it.
Stable reference material. Documentation, examples, a codebase summary.
Conversation history. Grows, but earlier turns do not change.
The newest message. Different every time.

On Claude, the API builds cache prefixes in the order tools, then system, then messages, and a change at one level invalidates the cache for that level and everything after it. Changing a tool definition invalidates everything. Changing the newest message invalidates nothing before it.

Prompt caching on Claude

On the Claude API you mark where cacheable content ends with a cache_control field. Everything up to and including that block becomes a cache entry. Per the prompt caching docs:

You can set up to 4 cache breakpoints per request.
There is also an automatic mode: a single top-level cache_control puts the breakpoint on the last cacheable block and moves it forward as the conversation grows.
The default cache lifetime is 5 minutes, refreshed for free each time the entry is used. A 1-hour lifetime is available with "ttl": "1h". The lifetime counts from the start of the request, so time spent generating the response counts against it.
Writing to the cache costs 1.25 times the base input price for the 5-minute lifetime and 2 times for the 1-hour lifetime. Reading from it costs 0.1 times the base price on most models, and less on some newer ones. Check the pricing table for your model.
There is a minimum cacheable length, which varies by model (from 512 to 4,096 tokens at the time of writing). Shorter prompts are simply not cached, with no error.

Here is an agent turn with two breakpoints: one at the end of the system prompt and one on the latest message, so each turn reuses everything before it.

import anthropic

client = anthropic.Anthropic()
MODEL = "your-claude-model-id"

TOOLS = []             # your tool definitions; keep this list identical on every turn
SYSTEM_PROMPT = "..."  # no timestamps or per-user text in here

def agent_turn(history: list[dict]) -> anthropic.types.Message:
    # Drop markers from earlier turns so only two breakpoints are ever sent.
    for msg in history:
        if isinstance(msg["content"], list):
            for block in msg["content"]:
                if isinstance(block, dict):
                    block.pop("cache_control", None)

    # Mark the newest message so the next turn can reuse this whole prefix.
    last = history[-1]
    if isinstance(last["content"], str):
        last["content"] = [{"type": "text", "text": last["content"]}]
    last["content"][-1]["cache_control"] = {"type": "ephemeral"}

    return client.messages.create(
        model=MODEL,
        max_tokens=2048,
        tools=TOOLS,
        system=[
            {
                "type": "text",
                "text": SYSTEM_PROMPT,
                "cache_control": {"type": "ephemeral"},
            }
        ],
        messages=history,
    )

The loop at the top removes the previous marker before adding a new one, so the request stays under the 4-breakpoint limit. That still hits the cache: the docs say the system looks back up to 20 blocks from each breakpoint for an earlier cache entry, so a breakpoint on the final block keeps working as long as each turn adds fewer than 20 blocks.

There is also a simpler option. The Messages API accepts a single top-level cache_control, and in that automatic mode the breakpoint moves forward with the conversation on its own.

Does it pay off?

Treat a normal, uncached prompt as costing 1.0 per use. With the 5-minute cache:

Two uncached uses cost 2.0.
One cache write plus one cache read costs 1.25 + 0.1 = 1.35.

You come out ahead the first time the prefix is reused. With the 1-hour cache, the write costs 2.0, so you need two reuses to come out ahead (2.0 + 0.1 + 0.1 = 2.2, against 3.0 uncached). For an agent that takes ten turns on one task, the shared prefix costs 1.25 + 9 × 0.1 = 2.15 instead of 10, a saving of almost 80 percent. These numbers use the standard 0.1x read price. On models with a lower read price, the savings are larger.

Prompt caching on OpenAI

OpenAI's prompt caching guide says caching is on by default for supported models. The default mode picks cache points for you, so there is nothing to mark. GPT-5.6 and later also let you place explicit breakpoints. A few things to know:

Prompts need at least 1,024 tokens to be cached on GPT-5.6 and later. On earlier models, the minimum varies with request settings.
The whole prefix has to match, so the same ordering rules apply.
On models before GPT-5.6, you can pass a stable prompt_cache_key to group requests that share a prefix, such as all requests for one user or one task. It helps routing but does not guarantee a hit. On GPT-5.6 and later, the guide says the key is not needed.
Retention options, write pricing and discounts vary by model generation, and newer models add more explicit controls. Check the guide for the model you use rather than relying on a number from a blog post, including this one.

The guide's own advice lines up with everything above: put stable instructions and reference material first, move dynamic content to the end, append to the conversation instead of rewriting earlier turns, and keep tool definitions stable.

Seven ways agents break their own cache

These are the common culprits when the hit rate is lower than it should be:

A timestamp in the system prompt. "Today is November 6, 2026, 3:42 PM" changes every minute. Put the date in the latest user message instead, or round it to the day.
Per-user text near the top. Move names, preferences and account details below the shared instructions.
Changing the tool list between turns. Adding or removing a tool invalidates everything on Claude. Keep the list fixed. On Claude, changing tool_choice still invalidates the cached messages, but it keeps the tools and system prompt cached, which costs far less than a full miss. OpenAI's guide recommends tool_choice: "none" over removing tool definitions.
Non-deterministic serialization. If tool schemas or context are built from a dictionary with unstable key order, the bytes change even when the content does not. Serialize with sorted keys.
Rewriting history. Summarizing or trimming earlier turns changes the prefix. When you must compact, do it rarely and in large steps, so you pay for one cache write instead of many.
Toggling settings. On Claude, changing tool_choice, the thinking settings or the effort setting, or adding or removing images, invalidates the cached messages.
Gaps longer than the lifetime. If a user pauses for ten minutes, a 5-minute entry is gone. For agents that wait on humans, the 1-hour lifetime can be worth its higher write price.

Measure your hit rate

Do not assume caching works. Read it from the response.

On Claude, the usage object reports three numbers: cache_creation_input_tokens (written to the cache), cache_read_input_tokens (served from it) and input_tokens (processed normally, after the last breakpoint). Their sum is your total input. If both cache fields are zero, nothing was cached, often because the prompt was under the minimum length.

On OpenAI's Responses API, cached tokens appear in usage.input_tokens_details.cached_tokens.

A simple hit rate for Claude:

def cache_hit_rate(usage) -> float:
    read = usage.cache_read_input_tokens or 0
    written = usage.cache_creation_input_tokens or 0
    fresh = usage.input_tokens or 0
    total = read + written + fresh
    return read / total if total else 0.0

Log it per turn. On a healthy multi-turn agent, the first turn shows mostly writes and later turns should be mostly reads. If later turns keep showing large writes, something near the top of your prompt is changing. Diff two consecutive requests and look for the first byte that differs.

Checklist

Tools first, then system prompt, then history, then the new message.
No timestamps, IDs or user details in the system prompt.
Tool list and schemas byte-for-byte identical across turns.
On Claude: a breakpoint on the system prompt and one on the latest message.
Hit rate logged on every call.

If you are cutting costs, it is also worth checking how much context your tools return on each call. Large tool responses sit in the history and get billed on every later turn. The post on why agents call the wrong tool covers how to keep them short.

FAQ

What is the minimum prompt length for Claude prompt caching?

It depends on the model, from 512 tokens on the newest models up to 4,096 on some older ones. Shorter prompts are processed without caching and return no error, and both cache fields in the usage object come back as 0.

How long does the Claude prompt cache last?

Five minutes by default, and each use refreshes it at no extra cost. For longer gaps, set "ttl": "1h", which costs 2 times the base input price to write instead of 1.25 times.

Why is cache_read_input_tokens always 0 on my Claude requests?

Common causes are a prompt under the model's minimum length, something before the breakpoint that changes on every request (such as a timestamp), a changed tool_choice, thinking setting or image, or a cache entry that expired.

Do I need to change my code to use OpenAI prompt caching?

No. Caching is on by default for supported models, and the default mode picks cache points for you. You can see hits in usage.input_tokens_details.cached_tokens on the Responses API.

Sources

Anthropic, Prompt caching (Claude API documentation)
OpenAI, Prompt caching (API documentation)