Skip to main content
How much a model reasons, whether it returns the reasoning text, which reasoning and caching fields it accepts, and how it caches prompts depend on the model family and the request format. This page covers the behavior; Models & Pricing has the rates.

Reasoning Effort

Set the effort in the field your request format uses. Use the field for the format you call, whatever the model’s provider. The formats recognize the values below, but each model supports a subset, and an effort the model does not support returns HTTP 400. On /messages, thinking also accepts {"type": "disabled"} and, on some models, {"type": "enabled", "budget_tokens": N}. Support varies by model: anthropic/claude-sonnet-5-5 rejects enabled with "thinking.type.enabled" is not supported for this model. Use "thinking.type.adaptive" and "output_config.effort" to control thinking behavior., and anthropic/claude-haiku-4-5 accepts enabled but rejects adaptive with adaptive thinking is not supported on this model. Values outside the list return HTTP 400, for example unsupported reasoning_effort value "pro". The Anthropic thinking field is accepted only on /messages; on /chat/completions it returns HTTP 400 invalid request body: json: unknown field "thinking". When you omit the effort, the model’s default applies. Setting an effort does not guarantee reasoning tokens, and omitting it does not guarantee none:
  • OpenAI models reason by default and stop when you set none. A trivial prompt can report 0 reasoning tokens with no setting too.
  • Anthropic models with adaptive thinking decide per request whether thinking is needed at the effort you set, so 0 reasoning tokens on a short prompt is normal.
  • GLM models reason by default, and none is served as the lowest effort rather than as off. At none, perplexity/glm-5.3 and perplexity/glm-5.3-flash report 0 reasoning tokens on most short prompts but can still produce a few.
Reasoning tokens are billed at the model’s output rate, whether or not the reasoning text is returned.

Reasoning Text

Each format reports reasoning text in its own place: message.reasoning_content on /chat/completions, reasoning output items on /responses (OpenAI summaries arrive in summary when you set reasoning.summary), and thinking content blocks on /messages. Whether any text comes back depends on the model family. GLM models return it whenever they reason, including at none. OpenAI and Anthropic models return it only when you set an effort other than none. Anthropic models return the thinking text; OpenAI models return a summary, and the summary is not always present even when reasoning tokens were used. When streaming, reasoning text arrives before the answer: as reasoning_content deltas in choices[0].delta on /chat/completions, as response.reasoning_summary_text.delta or response.reasoning_text.delta events on /responses (which one you get depends on the model and on whether the request sets reasoning, so handle both), and as content_block_delta events whose delta.type is thinking_delta on /messages. A model can return little or no reasoning text, so a quiet stream alone does not mean the connection stalled.

Carry Reasoning Across Turns

The Orchestrator API is stateless: send the full conversation on every turn instead of setting store or previous_response_id, which return HTTP 400 (see Errors). To keep a reasoning model’s earlier thinking in context, request it and send it back:
  • On /responses, set include: ["reasoning.encrypted_content"]. Each reasoning output item carries encrypted_content; send the item back, unchanged, in the next turn’s input along with the message items.
  • On /chat/completions, include reasoning_content on the assistant messages you send back. The text counts toward the next turn’s prompt tokens.
  • On /messages, send the assistant turn’s thinking content blocks back unchanged, with their signature, ahead of the text block.

Prompt Caching

Prompt caching works across requests, but how you turn it on depends on the model family. OpenAI models cache a repeated prefix automatically on /chat/completions and /responses. Repeats report the cached prefix in cached_tokens; models that bill cache writes report them in cache_write_tokens on the call that writes the cache. Short prompts are not cached: in our tests, a 639-token prompt reported no cache activity and a 1,259-token prompt did. Anthropic models cache only up to a marker at the end of the reusable prefix. Without a marker, they report no cache writes and no cache reads. On /messages, use cache_control as you would with the Anthropic Messages API. On /chat/completions, add prompt_cache_breakpoint to the last content part of the prefix:
On /responses, put the same prompt_cache_breakpoint on the last input_text part of the prefix. mode accepts only explicit. When the marked prefix is long enough to cache, the call that writes it reports cache_write_tokens and later calls report cached_tokens; a short prefix reports neither.

One-Hour Cache

By default, the marked prefix is cached for five minutes. Anthropic models can cache it for one hour: set prompt_cache_options to { "ttl": "1h" } on /chat/completions and /responses, or "ttl": "1h" inside cache_control on /messages. One-hour cache writes are billed at 2 times the model’s input rate. Other models reject "1h" with HTTP 400 prompt_cache_options.ttl "1h" is supported only on anthropic/* models, got model "perplexity/glm-5.3-flash".

Cache Misses

Each model keeps its own cache, and each deployment of a model caches independently. Cache markers alone do not pin a conversation to one deployment; send a stable x-session-id header or prompt_cache_key for best-effort affinity, as described under Provider Routing. A failover can still move a request and cause a cache miss, and with perplexity/auto a turn routed to a different model is not cached by the first. Prompt tokens that miss the cache are billed as fresh input at the model’s input rate. On models that charge for cache writes, tokens written to the cache are billed at the cache-write rate instead.

Where usage Reports Tokens

Each format reports cached and reasoning tokens in its own usage object. Responses do not include a cost field. One-hour cache writes appear in cache_write_tokens on Chat Completions and Responses, and in cache_creation.ephemeral_1h_input_tokens on Messages. The buckets overlap in the OpenAI formats. In Chat Completions and Responses, the total input count includes cache reads and cache writes, and the total output count includes reasoning tokens. To estimate cost, apply the model’s rates to each bucket, using the higher rate when the input exceeds the model’s context-length threshold: the input rate to total input minus cache reads and cache writes, the cache-read rate to cache reads, the cache-write rate to cache writes, and the output rate to total output. In Messages, input_tokens, cache_read_input_tokens, and cache_creation_input_tokens do not overlap, so each is charged at its own rate, and output_tokens includes thinking tokens. Rates are on Models & Pricing.

Next Steps

Models & Pricing

Per-token rates, cache-write premiums, and service tier multipliers.

Routing & Reliability

Model routing with perplexity/auto, deployment affinity, and failover.

FAQ

Why reasoning tokens are 0, why content is empty, and other common questions.