Skip to main content

Overview

The Orchestrator API routes at two levels:
  • Model routing is optional. Set model to perplexity/auto and a decision model picks the model for each request from the default policy or from routes you define.
  • Provider routing is automatic. Once the model is known, the platform picks a deployment of that model based on observed error rates, capacity, and latency, and retries on another deployment when one fails.
The response’s model field names the model that served the request. Billing uses that model’s published rates, regardless of which deployment served it.

Model Routing

Name a model from the catalog to call it directly. To let the API choose, set model to perplexity/auto. Omit routing (or set it to null) to use the default policy, or pass a routing object with your own routes. Model routing works on Chat Completions, Responses, and Messages. It is not supported on /messages/count_tokens, /responses/input_tokens, or /responses/compact. perplexity/auto is not listed by GET /orchestrator/v1/models.

How It Works

A routing object holds a default_model and one to five routes. Each route has a name, a description of the requests it should handle, and a model. For each request:
  1. Models that cannot serve the request are removed from the routes. The check covers the catalog and your usage tier, zero data retention and organization model settings, and the parameters and content the model accepts in the request format. If default_model is removed, the first remaining route’s model takes its place. If every route is removed, default_model serves without a routing decision. If no model remains, the request returns HTTP 400 with each model’s reason in the message. A route that names a model your tier cannot use is rejected before this step; see Route Rules.
  2. The decision model reads the conversation text and the route descriptions, not the route names or model IDs, and picks the route that best matches the task underway in the conversation, or none. When the user changes the task, it routes the new request.
  3. The picked route’s model serves the request. default_model serves when no route fits, when the decision model fails or times out, or when the conversation contains images, audio, video, or files.
The decision model sees system instructions, messages, tool-call names and arguments, and text tool results. Long conversations are truncated for the decision, keeping the system instructions, the first user request, and recent turns. The request sent to the selected model is not shortened. The compatibility check does not cover media, and it does not cover validation that only the provider performs, such as an unsupported JSON schema. Media sends the request to default_model, so choose one that accepts the media you send. Each request is routed on its own, so turns of one conversation can be served by different models. Send the full conversation on every request, as you would when naming a model directly. Each model keeps its own prompt cache; a turn routed to an Anthropic model is cached only when the request carries a cache marker. See Does Prompt Caching Work Across Requests?.

Default Policy

Without routing, perplexity/auto uses the default policy: openai/gpt-6.1-sol is the default model, and the simple route sends greetings, quick facts, titles and labels, and short rewrites or translations to openai/gpt-6-luna. Both models need usage Tier 1 or higher. At Tier 0, pass custom routes that name models your tier can use.
This request matched simple, so openai/gpt-6-luna served it. Your result can differ; read the headers and the model field to see which model answered:

Custom Routes

Pass routing as an object to define your own routes. Descriptions are the only route text the decision model sees, so describe the requests each route should handle. Routes may share a model, and a route may use the same model as default_model.
This request is written to match code_review and select anthropic/claude-sonnet-5-5. If the decision model returns none, openai/gpt-6-luna serves it. Check the response headers rather than assuming a match.

Route Rules

The serialized routing object must be at most 16,384 bytes. Unknown keys are rejected. Every model in routing is checked for catalog access at your usage tier when the request is parsed; a model your tier cannot use returns routing: model <id> is not available. Zero data retention, organization model settings, and the request’s parameters are checked per model in the compatibility step described in How It Works.

Response Headers

Successful responses carry an outcome header. The other headers depend on the decision: The headers are present on streaming responses too, since routing finishes before the stream starts. Requests rejected before routing return without them. An error raised after routing, such as a validation error from the served model, can still carry them.

Automatic Reasoning Effort

Add "reasoning_effort": "auto" to routing and the decision model also picks low, medium, or high effort for the request. The API writes the pick into the field the request format uses (reasoning_effort on Chat Completions, reasoning.effort on Responses, output_config.effort on Messages) before the served model sees the request. If no effort is selected or the served model does not support it, the model’s default effort applies and X-Perplexity-Autoroute-Effort reports default.
A request that sets its own effort beside "reasoning_effort": "auto" is rejected. This includes reasoning_effort on Chat Completions, reasoning.effort on Responses, and output_config.effort, a thinking budget, or disabled thinking on Messages. When routing omits reasoning_effort, your request’s effort setting passes through; if you omit both, the served model’s default applies.

Billing and Limits

You are billed at the served model’s published rates. The model field and the usage block describe the served model. Request-rate, concurrency, and balance checks run before the decision, so a request they reject never reaches the decision model. Token limits and request validation apply to the served model. For example, a Chat Completions request with max_tokens: 5 that routes to an OpenAI model returns Invalid 'max_output_tokens': integer below minimum value. Expected a value >= 16, but got 5 instead., the same as if you had named that model.

Errors

Routing validation errors return HTTP 400. On Chat Completions and Responses, the error has "type": "invalid_request_error" and "param": "routing", or "param": "model" when no model in routing is compatible with the request. Messages uses its native error envelope with type, error.type, error.message, and request_id, without param. Rate-limit, concurrency, and balance rejections, and errors from the selected model, keep their usual status and format. On Responses and Messages, the last message names reasoning.effort or output_config.effort or thinking in place of reasoning_effort.

Provider Routing

A model ID in the catalog can be served by any of several deployments. You do not choose the deployment, and there is no per-provider configuration. When a model has more than one deployment, traffic is split across them according to observed error rates, capacity, and latency. Deployments that degrade shed traffic to healthy ones and regain it as they recover. To keep the turns of one conversation on the same deployment, send a stable x-session-id header on every request, or a stable prompt_cache_key in the body on /chat/completions and /responses. Affinity is best effort: a failover can move the request to another deployment, and with perplexity/auto the header does not pin model selection across turns. See Does Prompt Caching Work Across Requests? for each model family’s cache markers.

Automatic Failover

When a deployment returns a provider error, a rate limit, or a timeout, the request is retried on another deployment of the same model when one is available. Each attempt is bounded by time-to-first-token and total-duration limits, and exceeding either counts as a failure.
  • An invalid request or a prompt that exceeds the model’s context window returns HTTP 400 and is not retried on another deployment.
  • When every deployment fails, the request returns HTTP 429 with a Retry-After header. Retrying after that interval is safe: requests that fail before producing any output are not billed.

Streaming

Failures before the first token are handled before the stream starts, so you receive a normal response. If a stream fails after output has begun, it ends with an in-band error event; treat an unterminated stream as incomplete. Streamed requests are billed for the tokens delivered.

Next Steps

Models & Pricing

The model catalog with per-token rates and usage tiers.

FAQ

Routing, session affinity, failover, and error handling questions.

API Reference

Request and response schemas and error shapes.