A source-backed guide to cacheable prefixes, GPT-5.6 breakpoints, retention, usage metrics, and the real cost of cache reads and writes.
· 9 min read
Loading article…
Find models, providers, pricing pages, and practical guides.
Prompt caching is easy to enable and surprisingly easy to model incorrectly. OpenAI enables it automatically for eligible requests, but a cache hit still depends on exact prefix reuse, routing, retention, and—on newer model families—the cost of writing that prefix in the first place.
This guide explains the current OpenAI behavior as of July 10, 2026. It separates OpenAI's official API behavior from the aggregated model-price index used elsewhere on LLM Price Lens.
Cache behavior and the OpenAI price examples below come from OpenAI's official documentation. The model directory on LLM Price Lens is indexed from the public OpenRouter Models API and should be treated as a comparison layer, not as an OpenAI invoice or contract.
Prompt caching reuses the model's work on an exact prompt prefix. OpenAI routes a request toward a machine that recently processed the same prefix, looks for a matching cached representation, and bills matching tokens at the cached-input rate. The model still generates a new answer; caching does not reuse an old completion.
The cacheable prefix can include more than plain text:
detail setting remain identicalThe practical rule is simple: if a byte of prompt structure changes before the end of the reusable prefix, later tokens move and the match can disappear.
Automatic prompt caching starts when the rendered prompt contains at least 1,024 tokens. Requests below that threshold still return a cached_tokens field, but its value is zero.
OpenAI normally routes requests using a hash derived from the beginning of the prompt—typically the first 256 tokens, although the exact length varies by model. A prompt_cache_key is combined with that prefix hash and gives you more control over routing requests that share the same long prefix.
This creates two separate requirements:
Eligibility alone is not proof of a hit. Always verify the usage response.
Put the slow-changing material first and the request-specific material last. A useful production ordering is:
Use a versioned cache key that represents the stable artifact, tenant, and policy version—not an individual request ID:
{
"model": "gpt-5.6",
"prompt_cache_key": "tenant:acme:knowledge-base-v3",
"input": [
{
"role": "user",
"content": [
{
"type": "input_file",
"file_id": "file_123",
"prompt_cache_breakpoint": { "mode": "explicit" }
},
{
"type": "input_text",
"text": "Answer today's customer question."
}
]
}
]
}
OpenAI recommends keeping total traffic across all prefixes for one key to roughly 15 requests per minute. Above that level, some requests may miss. High-volume systems should partition traffic across multiple stable keys while keeping the same prefix mapped to the same key.
Reusing prompt_cache_key improves locality, but the service still compares the exact prefix. The same key does not make two different prompts equivalent.
The behavior differs by model generation.
Earlier supported models use automatic prompt caching. Cache writes have no additional fee, and applications can select a retention policy with prompt_cache_retention when the model supports it.
Newer families support request-wide prompt_cache_options and explicit prompt_cache_breakpoint markers:
implicit is the default. OpenAI places a breakpoint on the latest message and also considers explicit markers.explicit disables that implicit marker. Only your explicit breakpoints are used; with no explicit marker, the request does not use prompt caching or incur cache-write charges.30m.For GPT-5.6 and later families, cache writes are reported in cache_write_tokens and billed at 1.25× the uncached input-token rate. Cache reads remain reported in cached_tokens and use the cached-input rate.
Explicit mode is useful when cache-write cost matters and you know exactly which stable prefixes are likely to be reused. It prevents the latest changing message from creating a write you did not intend.
OpenAI currently exposes two retention systems with different semantics.
| Model generation | Control | Meaning |
|---|---|---|
| GPT-5.6 and later families | prompt_cache_options.ttl | Minimum lifetime for breakpoints; currently 30m only and enabled by default |
| Earlier supported models | prompt_cache_retention | Maximum-retention policy such as in_memory or, where supported, 24h |
For earlier models using in_memory, cached prefixes generally remain active for 5–10 minutes after the last use and can remain for up to one hour. Extended retention can keep supported models' prefixes active for up to 24 hours by moving attention key/value tensors to GPU-local storage.
Do not choose retention only from expected traffic. Retention can affect data-control eligibility and defaults. OpenAI's current documentation says organizations with Zero Data Retention default to in-memory behavior on older models that support both policies, while other organizations default to 24-hour retention. Check the official Your data guide for the exact model, endpoint, region, and policy combination you use.
A good cache dashboard needs request-level counters, not a binary “cache enabled” flag. Log at least:
cached_tokenscache_write_tokens on GPT-5.6 and later familiesFor Chat Completions, OpenAI reports the counters under usage.prompt_tokens_details. Responses API exposes cached input details under usage.input_tokens_details.
{
"usage": {
"prompt_tokens": 2006,
"completion_tokens": 300,
"total_tokens": 2306,
"prompt_tokens_details": {
"cached_tokens": 1920,
"cache_write_tokens": 0
}
}
}
OpenAI's Standard, short-context pricing on July 10, 2026 lists these rates per one million tokens:
| Model | Input | Cached input | Cache write | Output |
|---|---|---|---|---|
| GPT-5.6 Luna | $1.00 | $0.10 | $1.25 | $6.00 |
| GPT-5.6 Terra | $2.50 | $0.25 | $3.125 | $15.00 |
| GPT-5.6 Sol | $5.00 | $0.50 | $6.25 | $30.00 |
Consider a stable 100,000-token prefix on GPT-5.6 Luna, ignoring dynamic input and output for clarity:
100,000 / 1,000,000 × $1.25 = $0.125100,000 / 1,000,000 × $0.10 = $0.012 × $0.10 = $0.20$0.125 + $0.01 = $0.135In this simplified case, the first successful reuse more than repays the $0.025 write premium. If no later request hits before eviction, the write was more expensive than ordinary input. That is why cache_write_tokens and reuse frequency belong in the same report.
Long-context, Batch, Flex, Priority, regional-processing, and third-party-hosted rates can differ. Re-read the pricing page before turning this example into a budget.
Treat cache keys as operational metadata. Keep customer text, credentials, and other secrets out of them, and review the current data-control documentation before using extended retention for sensitive workloads.
The following first-party sources define the OpenAI-specific claims in this article:
prompt_cache_key — request parameter contractThe LLM Price Lens model directory is generated from the public OpenRouter Models API. It normalizes prices for cross-provider comparison. When the OpenRouter index and an OpenAI billing decision disagree, use OpenAI's official pricing and account terms as the source of truth.