Skip to main content

Output Formats

formats chooses what each result includes besides metadata. The default is ["markdown"], and only the formats you request appear. An unknown format name returns 400 with error type unsupported_format. Highlights need a highlights object with your query, and max_tokens_per_url caps the length of each page’s highlights. One query applies to every URL in the request. Use highlights when your model needs only the parts of a page that answer a question.

Page Metadata

A result with status.type ok or not_modified has metadata, whatever formats asks for. To get only metadata, send "formats": []. metadata always has source_url, the URL the content came from after redirects. It can also have title, description, language, author, published_date, content_type, origin, fetched_at, and last_modified. Other meta tags in the page’s <head> with a non-empty name, property, or itemprop attribute and a non-empty value appear under their lowercased name, such as og:title or viewport: a string, or an array of the distinct values in page order if the page repeats the tag.

Freshness

By default, the API returns a stored copy of a page if it was fetched in the last two days (max_age_seconds 172800), and fetches the page live otherwise. metadata.origin shows which one you got: cached for a stored copy, or live for a page fetched for this request. metadata.fetched_at is when that content was fetched. A stored copy is returned without contacting the site, so that result can’t fail because the site is slow or down. Raise max_age_seconds to accept older copies, up to 63072000 (two years). Set it to 0 to always fetch the page live:

Stored Copies Only

Set cache_only to true to answer only from stored copies no older than max_age_seconds, without any live fetch. A URL with no stored copy that recent comes back with status.type no_content. cache_only needs max_age_seconds above 0; leave max_age_seconds out to use the two-day default.

Conditional Requests

To check whether a page changed since you last read it, send the metadata.last_modified value from your earlier response as if_modified_since. metadata.last_modified is present only when the site sends a Last-Modified header, and it is already in the required If-Modified-Since date format, such as Thu, 03 Dec 1992 08:37:20 GMT. A conditional request takes exactly one URL and max_age_seconds 0; any other date format, URL count, or max_age_seconds returns 400. If the page hasn’t changed, its result has status.type not_modified and metadata, but no content, so keep the content from your earlier request. If the site returns the page instead of 304, because it changed or the site doesn’t support conditional requests, the result is a normal ok result, and a failed fetch has its usual failure status. A not_modified result is billed like an ok result.

Timeouts

timeout_ms is the time budget for fetching the pages, 30000 by default; the response can take slightly longer than this budget. URLs are fetched in parallel. A URL that doesn’t finish in time comes back with status.type crawl_timeout, and the rest keep their content. For slow pages, raise timeout_ms and retry only the URLs that came back with crawl_timeout. The API reference lists the allowed range, which is lower with criteria.

Handle Errors

Per-URL Status

A request returns HTTP 200 even when some or all URLs fail, so check status.type on each result before you use its content. A failed result has only url and status, with a fixed status.message, and is not billed. status.status_code is the site’s own HTTP status, present on ok, not_modified, bot_challenge, and crawl_failed when the site answered. In this request, the first URL returns content, the site answers 404 for the second, and the third is not a URL:
Only the ok result is billed, so usage.fetched_urls is 1.

Request Errors

When the whole request fails, the response has an error object with message, type, and code, and no results. Errors about one field also have param, and errors the Context API returns itself also have request_id. A 401 means the API key is missing or invalid, and a 403 means your organization’s API access settings don’t allow this request. A 400 means the request needs a fix, because sending it again unchanged fails the same way. You can retry a 429 or a 5xx error; if the response has a Retry-After header, wait that many seconds first. The API reference lists the documented status codes and error types.

Next Steps

Score Pages with Criteria

Score each fetched page with yes or no questions and filter on the scores

API Reference

Request and response schemas for POST /v1/context

Reading List Q&A Cookbook

Fetch highlights for a question and answer it with numbered citations

Context API Pricing

Billing per fetched URL and for criteria scoring