Overview
The Orchestrator API routes at two levels:- Model routing is optional. Set
modeltoperplexity/autoand a decision model picks the model for each request from the default policy or from routes you define. - Provider routing is automatic. Once the model is known, the platform picks a deployment of that model based on observed error rates, capacity, and latency, and retries on another deployment when one fails.
model field names the model that served the request. Billing uses that model’s published rates, regardless of which deployment served it.
Model Routing
Name a model from the catalog to call it directly. To let the API choose, setmodel to perplexity/auto. Omit routing (or set it to null) to use the default policy, or pass a routing object with your own routes.
Model routing works on Chat Completions, Responses, and Messages. It is not supported on /messages/count_tokens, /responses/input_tokens, or /responses/compact. perplexity/auto is not listed by GET /orchestrator/v1/models.
How It Works
Arouting object holds a default_model and one to five routes. Each route has a name, a description of the requests it should handle, and a model. For each request:
- Models that cannot serve the request are removed from the routes. The check covers the catalog and your usage tier, zero data retention and organization model settings, and the parameters and content the model accepts in the request format. If
default_modelis removed, the first remaining route’s model takes its place. If every route is removed,default_modelserves without a routing decision. If no model remains, the request returns HTTP 400 with each model’s reason in the message. A route that names a model your tier cannot use is rejected before this step; see Route Rules. - The decision model reads the conversation text and the route descriptions, not the route names or model IDs, and picks the route that best matches the task underway in the conversation, or
none. When the user changes the task, it routes the new request. - The picked route’s model serves the request.
default_modelserves when no route fits, when the decision model fails or times out, or when the conversation contains images, audio, video, or files.
default_model, so choose one that accepts the media you send.
Each request is routed on its own, so turns of one conversation can be served by different models. Send the full conversation on every request, as you would when naming a model directly. Each model keeps its own prompt cache; a turn routed to an Anthropic model is cached only when the request carries a cache marker. See Does Prompt Caching Work Across Requests?.
Default Policy
Withoutrouting, perplexity/auto uses the default policy: openai/gpt-6.1-sol is the default model, and the simple route sends greetings, quick facts, titles and labels, and short rewrites or translations to openai/gpt-6-luna. Both models need usage Tier 1 or higher. At Tier 0, pass custom routes that name models your tier can use.
simple, so openai/gpt-6-luna served it. Your result can differ; read the headers and the model field to see which model answered:
Custom Routes
Passrouting as an object to define your own routes. Descriptions are the only route text the decision model sees, so describe the requests each route should handle. Routes may share a model, and a route may use the same model as default_model.
code_review and select anthropic/claude-sonnet-5-5. If the decision model returns none, openai/gpt-6-luna serves it. Check the response headers rather than assuming a match.
Route Rules
The serialized
routing object must be at most 16,384 bytes. Unknown keys are rejected. Every model in routing is checked for catalog access at your usage tier when the request is parsed; a model your tier cannot use returns routing: model <id> is not available. Zero data retention, organization model settings, and the request’s parameters are checked per model in the compatibility step described in How It Works.
Response Headers
Successful responses carry an outcome header. The other headers depend on the decision:
The headers are present on streaming responses too, since routing finishes before the stream starts. Requests rejected before routing return without them. An error raised after routing, such as a validation error from the served model, can still carry them.
Automatic Reasoning Effort
Add"reasoning_effort": "auto" to routing and the decision model also picks low, medium, or high effort for the request. The API writes the pick into the field the request format uses (reasoning_effort on Chat Completions, reasoning.effort on Responses, output_config.effort on Messages) before the served model sees the request. If no effort is selected or the served model does not support it, the model’s default effort applies and X-Perplexity-Autoroute-Effort reports default.
"reasoning_effort": "auto" is rejected. This includes reasoning_effort on Chat Completions, reasoning.effort on Responses, and output_config.effort, a thinking budget, or disabled thinking on Messages. When routing omits reasoning_effort, your request’s effort setting passes through; if you omit both, the served model’s default applies.
Billing and Limits
You are billed at the served model’s published rates. Themodel field and the usage block describe the served model. Request-rate, concurrency, and balance checks run before the decision, so a request they reject never reaches the decision model. Token limits and request validation apply to the served model. For example, a Chat Completions request with max_tokens: 5 that routes to an OpenAI model returns Invalid 'max_output_tokens': integer below minimum value. Expected a value >= 16, but got 5 instead., the same as if you had named that model.
Errors
Routing validation errors return HTTP 400. On Chat Completions and Responses, the error has"type": "invalid_request_error" and "param": "routing", or "param": "model" when no model in routing is compatible with the request. Messages uses its native error envelope with type, error.type, error.message, and request_id, without param. Rate-limit, concurrency, and balance rejections, and errors from the selected model, keep their usual status and format.
On Responses and Messages, the last message names
reasoning.effort or output_config.effort or thinking in place of reasoning_effort.
Provider Routing
A model ID in the catalog can be served by any of several deployments. You do not choose the deployment, and there is no per-provider configuration. When a model has more than one deployment, traffic is split across them according to observed error rates, capacity, and latency. Deployments that degrade shed traffic to healthy ones and regain it as they recover. To keep the turns of one conversation on the same deployment, send a stablex-session-id header on every request, or a stable prompt_cache_key in the body on /chat/completions and /responses. Affinity is best effort: a failover can move the request to another deployment, and with perplexity/auto the header does not pin model selection across turns. See Does Prompt Caching Work Across Requests? for each model family’s cache markers.
Automatic Failover
When a deployment returns a provider error, a rate limit, or a timeout, the request is retried on another deployment of the same model when one is available. Each attempt is bounded by time-to-first-token and total-duration limits, and exceeding either counts as a failure.- An invalid request or a prompt that exceeds the model’s context window returns HTTP 400 and is not retried on another deployment.
- When every deployment fails, the request returns HTTP 429 with a
Retry-Afterheader. Retrying after that interval is safe: requests that fail before producing any output are not billed.
Streaming
Failures before the first token are handled before the stream starts, so you receive a normal response. If a stream fails after output has begun, it ends with an in-band error event; treat an unterminated stream as incomplete. Streamed requests are billed for the tokens delivered.Next Steps
Models & Pricing
The model catalog with per-token rates and usage tiers.
FAQ
Routing, session affinity, failover, and error handling questions.
API Reference
Request and response schemas and error shapes.