Prompt Caching API Guide: Anthropic vs OpenAI (2025)

The Hidden Cost Eating Your LLM Budget

If you're calling large language models at any real volume, there's a cost driver that probably isn't getting enough attention: tokens you're sending over and over again.

A typical RAG pipeline stuffs several thousand tokens of document context into every request. A code review bot attaches the same style guide on every call. A customer support agent carries the same persona and rules through hundreds of daily conversations. None of that content changes between requests — but you're billed for every token, every single time.

Both Anthropic and OpenAI have prompt caching mechanisms designed to fix this. The implementations are meaningfully different, and using either one incorrectly means you're still paying full price. This guide breaks down how each works and gives you a concrete structure for maximizing cache hit rates.

---

How the Two Caching Systems Work

Anthropic: Explicit Markers, Precise Control

Claude Opus 4.8, Sonnet 4.6, and Haiku 4.5 use an explicit caching model. You mark which blocks you want cached by adding a cache_control field to your request:

response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": "You are a code review assistant. Below is the full style guide...",
            "cache_control": {"type": "ephemeral"}
        }
    ],
    messages=[{"role": "user", "content": user_question}]
)

Key rules to know:

• "type": "ephemeral" is the only supported cache type. TTL is 5 minutes — after that, the cache is gone.
• You can set up to 4 cache_control breakpoints per request.
• Minimum token thresholds apply: ≥ 1024 tokens for Opus 4.8, ≥ 2048 tokens for Sonnet 4.6 and Haiku 4.5. Fall short and nothing gets cached — no error, just no savings.
• Writing to cache costs 1.25× the standard input price. Reading from cache costs 0.1× — a 90% discount on every subsequent hit.

The usage object in the response includes cache_creation_input_tokens and cache_read_input_tokens, so you can monitor hit rates directly.

OpenAI: Automatic Caching, Zero Configuration

GPT-5.5, GPT-5.4, and GPT-5.4-mini use implicit caching. No extra fields needed — the system automatically caches prompt prefixes of ≥ 1024 tokens:

response = client.chat.completions.create(
    model="gpt-5.4",
    messages=[
        {"role": "system", "content": long_system_prompt},
        {"role": "user", "content": user_question}
    ]
)

# Check cache hits in usage
print(response.usage.prompt_tokens_details.cached_tokens)

What to keep in mind:

• Matching happens at 128-token prefix granularity — the prefix must be byte-for-byte identical to hit.
• TTL is roughly 5–10 minutes (shorter during low-traffic periods). OpenAI doesn't publish an exact figure.
• Cache hits are billed at approximately 50% of the standard input price — less aggressive than Anthropic's 90% discount.
• No manual control: you can't force a cache write or invalidate an entry.

Side-by-Side Comparison

	Anthropic	OpenAI
Cache type	Explicit cache_control	Automatic prefix matching
Minimum tokens	1024 / 2048	1024
Cache hit discount	90% off (0.1× price)	50% off (0.5× price)
TTL	5 minutes	5–10 minutes
Max breakpoints	4	Unlimited (automatic)
Monitoring field	cache_read_input_tokens	cached_tokens

---

Prompt Structure That Maximizes Cache Hits

The mechanisms differ, but both systems share one core principle: stable content goes first, dynamic content goes last.

Layer Your Content by Stability

Split your prompt into three tiers, ordered top to bottom:

1. Static (never changes): role definition, task instructions, output format, few-shot examples, reference documents
2. Semi-static (changes per session): conversation history
3. Dynamic (changes every request): the user's current input

For Anthropic, place your cache_control marker at the end of the static layer. For OpenAI, put your system message and fixed examples at the very front of the messages array and never touch their content between requests.

Put Few-Shot Examples in the Right Place

A common mistake is embedding few-shot examples inside the user message. On OpenAI's automatic caching, this shifts the prefix length on every turn and breaks cache matching entirely. The fix is to move examples into the system message, or place them as fixed turn pairs at the start of the messages array:

messages = [
    # Fixed few-shot turns — always first, never change
    {"role": "user", "content": "Example input A"},
    {"role": "assistant", "content": "Example output A"},
    {"role": "user", "content": "Example input B"},
    {"role": "assistant", "content": "Example output B"},
    # Dynamic content starts here
    *conversation_history,
    {"role": "user", "content": current_user_input}
]

Managing TTL in Low-Traffic Applications

Anthropic's 5-minute TTL is the main gotcha for low-frequency use cases. If your request interval drifts past 5 minutes, the cache expires and the next call triggers a cache write at 1.25× price again.

Two practical options:

• Heartbeat requests: Send a minimal request (1 token output) every 4 minutes to keep the cache warm. This has a cost, so it only makes sense when your cached content is >5,000 tokens.
• Factor in the write cost: If your traffic pattern means frequent cold starts, calculate whether the 1.25× write overhead still leaves you ahead on balance. Often it does.

Use Anthropic's Multiple Breakpoints for Layered Caching

The 4-breakpoint limit enables something useful: independent caching for content that changes at different rates. A legal assistant might have shared regulatory text (2,000 tokens) and per-user contract context (1,500 tokens):

system=[
    {
        "type": "text",
        "text": FIXED_LEGAL_RULES,      # Shared across all users
        "cache_control": {"type": "ephemeral"}
    },
    {
        "type": "text",
        "text": user_specific_contract, # Cached per user session
        "cache_control": {"type": "ephemeral"}
    }
]

The regulatory block gets cached once and shared across every user. The contract block gets cached per session. Both layers hit independently.

---

What the Savings Actually Look Like

Take a document Q&A app: 4,000-token system prompt, 50-token user questions, 300-token responses, 10,000 calls per day.

Without caching on Claude Sonnet 4.6: 4,050 tokens × 10,000 calls × $3/M = $121.50/day

With caching at an 80% hit rate: ~2,000 cold-start writes at 4,000 tokens × $3.75/M + ~8,000 cache reads at 4,000 tokens × $0.30/M + dynamic tokens = ~$31/day

That's roughly a 74% cost reduction — and hitting 80%+ is achievable as long as your request cadence stays within the TTL window and your prompt structure respects the static-first rule.

---

If you're doing this kind of API cost optimization, I'd recommend routing through XycAi. It's a single OpenAI-compatible endpoint covering 200+ models — GPT, Claude, Gemini, and more — with one API key and one-command setup for Claude Code, Codex, and Gemini CLI. Flagship models like GPT-5 and Claude Opus 4.8 are available at as low as 14% of list price, which compounds nicely on top of whatever caching savings you've already engineered.