Overview
Prompt caching reuses a prompt prefix that the model provider has already processed. Tokens read from the cache are billed at the model’s Cache read rate. Tokens written to the cache are billed at its Cache write rate, which is above the input rate for some models. See Models for both rates. By default, the Agent API places cache breakpoints on the system instructions and on the latest message, and the provider may also cache automatically. You don’t need to change anything to benefit from caching within one agent run.Billing
Every response reports cache usage inusage:
Reuse a prefix across requests
Send the sameprompt_cache_key on every request that shares a prefix, including retries and multi-turn conversations. Requests with the same key are routed for cache affinity, so a later request can read the prefix an earlier one wrote.
- The key is scoped to your account. Other accounts never share your cache affinity, even if they use the same key.
- The key does not change which model handles the request.
- Use one key per workload that shares a prefix, such as one key per document or prompt template. Unrelated workloads should use different keys.
- Effort levels set a default key for their shared system prompt and tools. A key you send overrides it.
- Put content that stays the same (instructions, documents, schemas) at the start of the prompt and content that changes at the end. A cache read needs an exact prefix match.
previous_response_id. Earlier turns reach the model unchanged, so the new request can read the prefix the previous one wrote.
Control what is cached
Useprompt_cache_options.mode and prompt_cache_breakpoint to choose which prefixes are written to the cache:
Put a mark on the
input_text part that ends a reusable prefix. Everything before the mark, including the instructions and earlier messages, is cached with it. Marks are saved with the response, so a previous_response_id continuation keeps them; if a continuation ends up with more marks than its mode allows, the newest ones are used.
The API returns HTTP 400 if a request has more marks than its mode allows or marks a part other than input_text. prompt_cache_options.ttl is accepted for OpenAI compatibility and ignored: providers apply their default cache lifetime.
The example marks only the stable document, so the changing question is never written to the cache:
Next Steps
Agent API Models
Compare input, cache read, and cache write rates.
API Reference
View complete endpoint documentation.