> ## Documentation Index
> Fetch the complete documentation index at: https://docs.perplexity.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Use Decisions API grounded web search to fact check claims

> Fact check claims with the Decisions API in web_grounding mode. Put the claim first in each state so the search sees it, read search_results, and let your code turn the probability the model returns into a supported, contradicted, or review verdict.

Ask a decision model whether Navi Pillay was awarded the 2026 Nobel Peace Prize and it returns `0.01`. She was. Without a search the model answers from the `state` alone, and the `state` says nothing about last month. With `mode: "web_grounding"` the same request returned `1.00`. Put 600 characters of unrelated text in front of the claim and the grounded request returned `0.07`, with five office-move pages in `search_results`: the query is built from the start of the `state`, and the claim was not in it. This recipe shows how to shape the request so the search sees your claim, how to confirm it did, and how to measure what grounding changes.

A decision model returns a probability, not prose. Without `mode`, it answers from the `state` you send alone. With `mode: "web_grounding"`, the API builds a search query from the question text and the start of your `state`, runs a [Fast Search](https://docs.perplexity.ai/docs/search/fast-search.md), appends the results to the `state` as evidence, and the model answers from that. The response adds `search_results`, the evidence the model saw. Your code compares the probability to two cutoffs and marks the claim `supported`, `contradicted`, or `review`.

The recipe runs 24 labeled claims, 8 about facts settled for years and 16 about events from September and October 2026; without grounding no true recent claim printed above `0.24`, and with it 24 of 24 came back right.

<video autoPlay muted loop playsInline controls src="https://mintcdn.com/perplexity/qqD-p40BwDQJUMhc/docs/assets/images/cookbook/examples/decisions-api-web-grounding-demo.mp4?fit=max&auto=format&n=qqD-p40BwDQJUMhc&q=85&s=8ec2cb1306295eb989c7b9e174502d49" />

## Why web grounding

* **One call instead of two APIs stitched together.** One `POST /v1/decisions` replaces a Search API request, prompt assembly, and a second request.
* **A number you can threshold.** `p >= 0.80` is supported, `p <= 0.20` is contradicted, and the middle goes to a person. The cutoffs are constants in your code, not instructions to the model.
* **The search sees the start of your `state`.** The query is the question text and then as much of your `state` as fits in 256 bytes, so where you put the claim decides what the search looks for, and `search_results` shows you what it found.

## What you need

* Python 3.10 or newer on macOS or Linux. The shell commands use Unix syntax.
* A Perplexity API key. Every key can call the Decisions API. A grounded request bills input tokens as usual plus \$1.00 per 1,000 searches.

## Set up

Everything lives in one directory. You create the first six files from this page; the script writes `out/` on every run.

The script posts to `https://api.perplexity.ai/v1/decisions` with `httpx`, like the [ticket triage recipe](https://docs.perplexity.ai/docs/cookbook/examples/decisions-api-ticket-triage/README.md), so the exact body the gateway reads is on the page. With the Perplexity Python library (0.46.1), which has no named `mode` parameter or `search_results` field yet, pass `extra_body={"mode": "web_grounding"}` to `client.decisions.create` and read the results from `response.to_dict()["search_results"]`.

```text theme={null}
decisions-api-web-grounding/
├── requirements.txt       # two pinned dependencies
├── claims.jsonl           # 24 labeled claims, one JSON object per line
├── query.py               # the gateway's search query rule, mirrored
├── factcheck.py           # check and eval
├── test_query.py          # the gateway's own test cases, offline
├── test_factcheck.py      # offline tests, no network
├── out/                   # one JSON file per run, written by the script
└── .venv/                 # virtual environment from the install step
```

> [accordion] requirements.txt
>
> ```text requirements.txt theme={null}
> httpx==0.28.1
> pytest==9.1.1
> ```

Create the directory, save `requirements.txt` in it, then create a virtual environment, install, and export your key.

> [accordion] Install and set the key
>
> ```bash theme={null}
> mkdir decisions-api-web-grounding && cd decisions-api-web-grounding
> # save requirements.txt here, then continue
> python3 -m venv .venv
> source .venv/bin/activate
> python -m pip install -r requirements.txt
> export PERPLEXITY_API_KEY="your-api-key-here"
> ```

### The claims

Each line has an `id`, the `claim`, a `label` (`true` or `false`, checked against the linked `source`), a `kind`, and the `source` URL. `stable` claims are facts that have not changed in years. `recent` claims are about events from September and October 2026: Nobel Prizes, the World Series, the Ballon d'Or, a Python release, a Federal Reserve meeting. 13 are true and 11 are false. The script sends only the `claim` text; the label and the source stay on your side. The longest claim is 100 bytes, so with the 18-byte question every claim fits the query in full when it comes first.

> [accordion] claims.jsonl
>
> ```json claims.jsonl theme={null}
> {"id": "s01", "claim": "Python 3.12 was released in October 2023.", "label": true, "kind": "stable", "source": "https://www.python.org/downloads/release/python-3120/"}
> {"id": "s02", "claim": "Rust 1.0 was released in May 2015.", "label": true, "kind": "stable", "source": "https://blog.rust-lang.org/2015/05/15/Rust-1.0.html"}
> {"id": "s03", "claim": "The Great Barrier Reef lies off the coast of Queensland, Australia.", "label": true, "kind": "stable", "source": "https://en.wikipedia.org/wiki/Great_Barrier_Reef"}
> {"id": "s04", "claim": "HTTP status code 404 means the requested resource was not found.", "label": true, "kind": "stable", "source": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404"}
> {"id": "s05", "claim": "The Linux kernel was first released in 1998.", "label": false, "kind": "stable", "source": "https://en.wikipedia.org/wiki/Linux_kernel"}
> {"id": "s06", "claim": "Mount Kilimanjaro is the highest mountain in South America.", "label": false, "kind": "stable", "source": "https://en.wikipedia.org/wiki/Mount_Kilimanjaro"}
> {"id": "s07", "claim": "Git was created by Guido van Rossum.", "label": false, "kind": "stable", "source": "https://en.wikipedia.org/wiki/Git"}
> {"id": "s08", "claim": "PostgreSQL is licensed under the GNU General Public License.", "label": false, "kind": "stable", "source": "https://www.postgresql.org/about/licence/"}
> {"id": "r01", "claim": "Navi Pillay was awarded the 2026 Nobel Peace Prize.", "label": true, "kind": "recent", "source": "https://www.nobelprize.org/prizes/peace/2026/summary/"}
> {"id": "r02", "claim": "Francis Halzen won the 2026 Nobel Prize in Physics for the IceCube Neutrino Observatory.", "label": true, "kind": "recent", "source": "https://www.nobelprize.org/prizes/physics/2026/summary/"}
> {"id": "r03", "claim": "Canadian author Anne Carson won the 2026 Nobel Prize in Literature.", "label": true, "kind": "recent", "source": "https://www.nobelprize.org/prizes/literature/2026/press-release/"}
> {"id": "r04", "claim": "Henri Kagan and Kenso Soai shared the 2026 Nobel Prize in Chemistry.", "label": true, "kind": "recent", "source": "https://www.nobelprize.org/prizes/chemistry/2026/summary/"}
> {"id": "r05", "claim": "The Milwaukee Brewers won the 2026 World Series, the first title in franchise history.", "label": true, "kind": "recent", "source": "https://www.mlb.com/news/brewers-win-world-series-2026"}
> {"id": "r06", "claim": "Python 3.15.0 was released on October 9, 2026.", "label": true, "kind": "recent", "source": "https://docs.python.org/3/whatsnew/3.15.html"}
> {"id": "r07", "claim": "Harry Kane won the 2026 men's Ballon d'Or.", "label": true, "kind": "recent", "source": "https://www.espn.com/soccer/story/_/id/50236983/harry-kane-2026-ballon-dor-kylian-mbappe-lamine-yamal"}
> {"id": "r08", "claim": "The Federal Reserve held its target range at 3.75 to 4 percent at its October 28, 2026 meeting.", "label": true, "kind": "recent", "source": "https://www.federalreserve.gov/newsevents/pressreleases/monetary20261028a.htm"}
> {"id": "r09", "claim": "Haruki Murakami won the 2026 Nobel Prize in Literature.", "label": false, "kind": "recent", "source": "https://www.nobelprize.org/prizes/literature/2026/press-release/"}
> {"id": "r10", "claim": "The Tampa Bay Rays won the 2026 World Series.", "label": false, "kind": "recent", "source": "https://www.mlb.com/news/brewers-win-world-series-2026"}
> {"id": "r11", "claim": "Lamine Yamal won the 2026 men's Ballon d'Or.", "label": false, "kind": "recent", "source": "https://www.espn.com/soccer/story/_/id/50236983/harry-kane-2026-ballon-dor-kylian-mbappe-lamine-yamal"}
> {"id": "r12", "claim": "The Federal Reserve cut its target rate by a quarter point at its October 28, 2026 meeting.", "label": false, "kind": "recent", "source": "https://www.federalreserve.gov/newsevents/pressreleases/monetary20261028a.htm"}
> {"id": "r13", "claim": "Apple unveiled the iPhone 18 Pro at an event in October 2026.", "label": false, "kind": "recent", "source": "https://en.wikipedia.org/wiki/IPhone_18_Pro"}
> {"id": "r14", "claim": "The 2026 Nobel Prize in Physics was awarded for the discovery of room-temperature superconductivity.", "label": false, "kind": "recent", "source": "https://www.nobelprize.org/prizes/physics/2026/summary/"}
> {"id": "r15", "claim": "Daron Acemoglu and Simon Johnson won the 2026 Nobel Prize in Economic Sciences.", "label": false, "kind": "recent", "source": "https://www.reuters.com/world/hall-woodford-win-2026-nobel-economics-prize-2026-10-12/"}
> {"id": "r16", "claim": "Karl Deisseroth, Peter Hegemann and Georg Nagel won the 2026 Nobel Prize in Physiology or Medicine.", "label": true, "kind": "recent", "source": "https://www.nobelprize.org/prizes/medicine/2026/summary/"}
> ```

## Shape the request

Before the model sees anything, the gateway builds one search query per question. The query is the question's `instructions` (and its criteria text, if you wrote any), then the text of your `state`, in that order, cut to 256 bytes of normalized UTF-8. When the `state` has text, the question part gets at most 128 bytes; the separator takes one more, so a question that uses all 128 leaves 127 for the `state`. The `state` part is a prefix: the first bytes of your `state`, as many as fit. A word the limit would cut is dropped unless at least 32 bytes of it fit, so an unspaced script keeps its start, and once something is cut nothing later fills the gap. Runs of whitespace and control characters collapse to one space, the ends are trimmed, a code point is never split, and `data:` URLs are skipped. The search runs with that query, the gateway appends the results to a string `state` after an `Evidence:` line, the model answers, and the response carries the same results in `search_results`: title, URL, snippet, and dates, or `[]` when nothing was added.

So keep the question short, because every byte of question text is a byte the claim cannot use: "Is the claim true?" is 18 bytes, which leaves 237 for the claim. And send the claim alone, or keep it at the start, before any other context. Text in front of the claim uses up those bytes: once it is long enough, the limit cuts the claim mid-sentence and then drops it altogether, and the search runs on the text in front instead. [What the search sees](#what-the-search-sees) shows what that did here.

`query.py` is a port of the gateway's rule for the shape this recipe sends: a string `state` and one `noul` question with plain `instructions`. `search_query` gives you the query text, and `claim_in_query` tells you before sending whether the complete claim text is inside it; a claim the limit cuts in the middle counts as outside. The script runs that check on every claim, prints the count, and warns when any claim is outside. For criteria, `choice` and `score` questions, and object or array `state`, the helper raises instead of guessing; [Web grounding](https://docs.perplexity.ai/docs/decisions/quickstart.md#web-grounding) has the rule for those.

### The query rule

`normalize_prefix` is the normalization above, applied to one part with a byte budget; the second value it returns says whether the input was truncated. `Text` accumulates parts up to a budget and, after a cut, pins its budget at the current length so a later part cannot fill a dropped word's gap. `search_query` runs the `state` through a 256-byte `Text`, the question text through a 128-byte one when the `state` has text, and joins the two in a final 256-byte `Text`. It counts UTF-8 bytes, so a claim with accented characters fits fewer characters than one in plain ASCII. `is_data_url` trims the same characters Go's `TrimSpace` does, which is narrower than Python's `strip`. `claim_in_query` normalizes the claim the same way and looks for the whole of it in the query.

> [accordion] query.py
>
> ```python query.py theme={null}
> """The search query the gateway builds for pplx-decider web grounding, mirrored for the shapes this recipe
> sends. The rule: one query per question, the question text first and then the start of the `state`, 256
> bytes in all, whitespace-normalized and cut at word boundaries; with `state` text the question part gets
> at most 128 bytes. The rule is documented under Web grounding in the Decisions quickstart
> (docs/decisions/quickstart#web-grounding)."""
>
> import unicodedata
>
> MAX_QUERY_BYTES = 256  # the Search API per-query limit: question text, then the start of the state
> MAX_DROPPED_CUT = 32  # a word cut by the limit is dropped when fewer bytes than this fit
>
>
> def normalize_prefix(s: str, n: int) -> tuple[str, bool]:
>     """The gateway's normalization: whitespace and control runs become one space, ends trimmed, at most n
>     bytes without splitting a code point. A word cut by the limit is dropped when fewer than MAX_DROPPED_CUT
>     bytes of it fit. The flag reports that s did not fit."""
>     out, pending, cut, truncated, last_word = bytearray(), False, False, False, 0
>     for char in s:
>         sep = char.isspace() or unicodedata.category(char) == "Cc"
>         if len(out) >= n or (pending and out and len(out) + 2 > n):
>             cut, truncated = not sep and not pending, True
>             break
>         if sep:
>             pending = True
>             continue
>         need = len(char.encode("utf-8")) + (1 if pending and out else 0)
>         if len(out) + need > n:
>             cut, truncated = not pending, True
>             break
>         if pending and out:
>             last_word = len(out)
>             out += b" "
>         pending = False
>         out += char.encode("utf-8")
>     written = len(out) - last_word - (1 if last_word else 0)
>     if cut and written < MAX_DROPPED_CUT:
>         return out[:last_word].decode("utf-8"), True
>     return out.decode("utf-8"), truncated
>
>
> class Text:
>     """Normalized words up to max bytes. After a cut, later parts cannot fill the gap: it stays a prefix."""
>
>     def __init__(self, max_bytes: int) -> None:
>         self.s, self.max = "", max_bytes
>
>     def room(self) -> int:  # bytes another word may take, after its separator
>         used = len(self.s.encode("utf-8"))
>         return self.max - used - (1 if used else 0)
>
>     def add(self, s: str) -> None:
>         if self.room() <= 0:
>             return
>         part, truncated = normalize_prefix(s, self.room())
>         if part:
>             self.s = f"{self.s} {part}" if self.s else part
>         if truncated:
>             self.max = len(self.s.encode("utf-8"))
>
>
> GO_SPACE = ("\t\n\v\f\r \x85\xa0\u1680\u2000\u2001\u2002\u2003\u2004\u2005\u2006\u2007\u2008\u2009\u200a"
>             "\u2028\u2029\u202f\u205f\u3000")  # Go's unicode.IsSpace, what strings.TrimSpace removes
>
>
> def is_data_url(s: str) -> bool:
>     """The gateway's check: after Go's TrimSpace set (not Python's wider strip), the text starts with data:."""
>     return s.strip(GO_SPACE)[:5].lower() == "data:"
>
>
> def search_query(state: str, question: dict) -> str:
>     """The search text the gateway builds: the question text, then the start of the state, MAX_QUERY_BYTES in
>     all. With state text the question gets at most half. Ports the gateway for a string state and one
>     `noul` question with string instructions and no criteria; other shapes raise instead of guessing."""
>     text = question.get("instructions")
>     if (question.get("type") != "noul" or "criteria" in question or not isinstance(text, (str, type(None)))
>             or not isinstance(state, str)):
>         raise ValueError("search_query covers a string state and a noul question with string instructions only")
>     state_part = Text(MAX_QUERY_BYTES)
>     if not is_data_url(state):
>         state_part.add(state)
>     question_part = Text(MAX_QUERY_BYTES // 2 if state_part.s else MAX_QUERY_BYTES)
>     if text and not is_data_url(text):
>         question_part.add(text)
>     query = Text(MAX_QUERY_BYTES)
>     query.add(question_part.s)
>     query.add(state_part.s)
>     return query.s
>
>
> def claim_in_query(claim: str, state: str, question: dict) -> bool:
>     """True when the complete claim text, normalized the same way, appears in the search query built from state.
>     A claim the limit cuts in the middle counts as outside, even though its start is in the query."""
>     return normalize_prefix(claim, len(claim.encode("utf-8")))[0] in search_query(state, question)
> ```

The tests for it are the gateway's own cases, ported: every `normalizePrefix` assertion, the prefix-after-a-cut case, the half and whole budgets, the start of a long `state`, an unspaced Japanese `state`, `data:` URLs in either position and behind a control character, the shapes the port refuses, and where a padded claim stops being inside the query, including the padding that keeps the claim's start and cuts its end.

> [accordion] test\_query.py
>
> ```python test_query.py theme={null}
> """Offline checks for query.py, ported from the gateway's own test cases. Run: python -m pytest -q"""
>
> import pytest
>
> from factcheck import QUESTION, pad
> from query import MAX_DROPPED_CUT, MAX_QUERY_BYTES, Text, claim_in_query, normalize_prefix, search_query
>
> LONG = "word " * 200
> WORDS = {"type": "noul", "instructions": LONG}
>
>
> @pytest.mark.parametrize(("s", "n", "expected", "truncated"), [
>     ("  a \t\n b  ", 100, "a b", False),
>     ("a  b", 2, "a", True),  # a separator that would overflow stops the output
>     ("é é", 3, "é", True),  # never split a code point
>     ("   ", 10, "", False),
>     ("a" + " " * (1 << 20) + "b", 2, "a", True),
>     (" b", 1, "b", False),
>     ("a\x00\x1b[0m\x7fb", 100, "a [0m b", False),  # control characters separate words
>     ("alpha shipping", 10, "alpha", True),  # a word cut by the limit is dropped
>     ("ab cd", 4, "ab", True),  # a word that reaches the limit mid-word is dropped
>     ("ab cd e", 5, "ab cd", True),  # a complete last word is kept
>     ("shipping news", 4, "", True),  # a short fragment is dropped even when it is the only word
>     ("a b", 3, "a b", False),
>     ("a " + "x" * 40, MAX_DROPPED_CUT + 2, "a " + "x" * MAX_DROPPED_CUT, True),  # a long cut word keeps a prefix
> ])
> def test_normalize_prefix_matches_the_gateway(s: str, n: int, expected: str, truncated: bool) -> None:
>     assert normalize_prefix(s, n) == (expected, truncated)
>
>
> def test_text_stays_a_prefix_after_a_cut() -> None:
>     t = Text(12)
>     t.add("Order shipping")
>     t.add("late")
>     assert t.s == "Order"  # a later part must not fill the dropped word's gap
>
>
> def test_query_is_capped_at_the_search_limit() -> None:
>     query = search_query(LONG, WORDS)
>     assert len(query.encode()) <= MAX_QUERY_BYTES and query.startswith("word word") and not query.endswith(" ")
>     half, whole = MAX_QUERY_BYTES // 2 // len("word "), MAX_QUERY_BYTES // len("word ")
>     assert search_query("Order", WORDS) == ("word " * half).strip() + " Order"  # the question keeps half
>     assert search_query("", WORDS) == ("word " * whole).strip()  # no state: the whole budget
>     microsoft = search_query("Microsoft fiscal 2025", WORDS)
>     assert microsoft != search_query("Apple fiscal 2025", WORDS) and microsoft.endswith(" Microsoft fiscal 2025")
>
>
> def test_query_searches_the_start_of_a_long_state() -> None:
>     text = "Did Microsoft's fiscal 2023 revenue exceed $240 billion?"
>     trajectory = "step: the agent read a page and wrote notes. " * 50
>     query = search_query(trajectory, {"type": "noul", "instructions": text})
>     assert query.startswith(text + " step: the agent read a page and wrote notes.") and len(query.encode()) <= 256
>     state_only = search_query(trajectory, {"type": "noul"})  # no question text: the state
>     assert state_only == trajectory.strip()[:len(state_only)] and 200 < len(state_only.encode()) <= 256
>
>
> def test_query_keeps_an_unspaced_state_prefix() -> None:
>     state = "注文番号12は配送が遅れました" * 20
>     query = search_query(state, {"type": "noul", "instructions": "Was the order late?"})
>     assert len(query.encode()) <= 256 and query.startswith("Was the order late? " + state[:14])
>
>
> @pytest.mark.parametrize(("state", "question", "expected"), [
>     ("  Order   #12\nshipped late ", {"type": "noul", "instructions": "Was it late?"},
>      "Was it late? Order #12 shipped late"),
>     ("data:text/plain,claim", QUESTION, QUESTION["instructions"]),  # a data: state is skipped
>     ("claim", {"type": "noul", "instructions": " DATA:text/plain,q"}, "claim"),  # so is a data: question
>     ("claim", {"type": "noul", "instructions": None}, "claim"),
>     ("\x1cdata:text/plain,claim", QUESTION, "Is the claim true? data:text/plain,claim"),  # \x1c is not TrimSpace
>     ("\u3000data:text/plain,claim", QUESTION, QUESTION["instructions"]),  # ideographic space is
> ])
> def test_search_query_normalizes_and_skips_data_urls(state: str, question: dict, expected: str) -> None:
>     assert search_query(state, question) == expected
>
>
> @pytest.mark.parametrize(("state", "question"), [
>     ("x", {"type": "noul", "instructions": "Is it true?", "criteria": {"true": "yes", "false": "no"}}),
>     ("x", {"type": "noul", "instructions": {"ask": "Is it true?"}}),
>     ("x", {"type": "choice", "instructions": "Pick one.", "options": ["a", "b"]}),
>     ("x", {"type": "score", "instructions": "Rate it."}),
>     ("x", {"instructions": "Is it true?"}),  # no type
>     ("x", {"type": "NOUL", "instructions": "Is it true?"}),  # the gateway matches the type exactly
>     (["x"], QUESTION)])
> def test_search_query_refuses_shapes_it_does_not_port(state, question: dict) -> None:
>     with pytest.raises(ValueError):
>         search_query(state, question)
>
>
> @pytest.mark.parametrize(("state", "inside"), [
>     ("Python 3.12 was released in October 2023.", True),
>     ("Python  3.12 was released\nin October 2023.", True),  # the claim is normalized the same way
>     (pad("Python 3.12 was released in October 2023.", 0, 600), True),  # claim first: filler follows it
>     (pad("Python 3.12 was released in October 2023.", 150, 0), True),  # 18 + 1 + 150 + 1 + 41 bytes fit
>     (pad("Python 3.12 was released in October 2023.", 200, 0), False),  # the limit cuts the claim's last word
>     (pad("Python 3.12 was released in October 2023.", 600, 0), False),  # the claim starts past the limit
> ])
> def test_claim_in_query_depends_on_where_the_claim_sits(state: str, inside: bool) -> None:
>     assert claim_in_query("Python 3.12 was released in October 2023.", state, QUESTION) is inside
>
>
> def test_a_cut_claim_keeps_only_its_start() -> None:
>     query = search_query(pad("Python 3.12 was released in October 2023.", 200, 0), QUESTION)
>     assert query.endswith(" Python 3.12 was released in October") and len(query.encode()) <= 256
>     assert search_query(pad("Python 3.12 was released in October 2023.", 600, 0), QUESTION).find("Python") == -1
> ```

### Constants and the padding helper

`QUESTION` is the one question every request asks. The cutoffs and the three acceptance bounds are the policy; the model never sees them. `PRICE_PER_M_INPUT` and `PRICE_PER_SEARCH` are the published rates, used to turn `usage.input_tokens` and the request count into a cost per 1,000 claims. `FILLER` is the office-move text the padding experiments put around each claim; it is in the file so the experiments are reproducible. `pad` builds the experiment input: the claim on its own line after `before` characters of `FILLER` and before `after` more.

> [accordion] Constants and pad()
>
> ```python factcheck.py (part 1 of 4) theme={null}
> """Fact-check claims against the web with the Decisions API. One request per claim: the claim is the
> `state`, one `noul` question asks whether it is true, and `mode: "web_grounding"` adds a web search."""
>
> import argparse
> import json
> import os
> import sys
> import time
> from dataclasses import asdict, dataclass, field
> from pathlib import Path
>
> import httpx
>
> from query import claim_in_query
>
> DECISIONS_URL = "https://api.perplexity.ai/v1/decisions"
> MODEL = "pplx-decider-latest"  # selects pplx-decider-v1.2
> QUESTION_NAME = "is_true"
> QUESTION = {"type": "noul", "instructions": "Is the claim true?"}
>
> SUPPORTED_MIN = 0.80  # p >= this: supported. Your code applies the cutoffs, not the model.
> CONTRADICTED_MAX = 0.20  # p <= this: contradicted; anything between: review
> MAX_MISSED_FAILURES = 0  # acceptance bounds the eval reports against: false claims marked supported
> MAX_FALSE_CONTRADICTED_RATE = 0.05  # true claims marked contradicted, as a share of claims
> MAX_REVIEW_RATE = 0.25  # share of claims sent to a person
>
> PRICE_PER_M_INPUT = {"pplx-decider-latest": 0.02, "pplx-decider-v1.2": 0.02,  # $ per million input tokens
>                      "pplx-decider-flash-latest": 0.01, "pplx-decider-flash-v1.2": 0.01}
> PRICE_PER_SEARCH = 0.001  # dollars per web search when grounded
> RETRIES = 3  # on 429 and 5xx; waits Retry-After seconds when it is an integer, else 1, 2, 4
> MAX_WAIT_S = 30  # cap on any single wait
> OUT_DIR = Path("out")
>
> FILLER = (  # neutral text for --pad-before and --pad-after; nothing in it relates to any claim
>     "The facilities team confirmed the office move for the second week of the quarter. "
>     "Badge readers on the third floor will be replaced on Monday, and visitor parking "
>     "moves to the lower level. The updated seating chart goes out by email once the "
>     "furniture vendor confirms delivery. Please label boxes with your team name. ")
>
>
> def pad(claim: str, before: int, after: int) -> str:
>     """The claim on its own line, after `before` characters of FILLER and before `after` more."""
>     def filler(n: int) -> list[str]:
>         return [(FILLER * (n // len(FILLER) + 1))[:n]] if n else []
>     return "\n".join(filler(before) + [claim] + filler(after))
> ```

### One request per claim

`decide` sends the claim as `state` with the one question, and adds `"mode": "web_grounding"` only when asked. `429` and `5xx` responses are retried up to three times, waiting `Retry-After` seconds when the header is a whole number and 1, 2, then 4 seconds when it is absent or anything else, never more than 30 seconds. Any other error raises `DecisionsError` with the status and `error.message` from the body, and so does a `200` whose body is not the expected shape, including a probability outside `[0, 1]`, a negative token count, or a search result without a `url`. The `Decision` it returns holds the probability from `answers.is_true.noul`, `usage.input_tokens`, the wall time for the claim with retries included, the number of POSTs it took, and the `search_results` URLs, `[]` when the key is absent. The POST count matters when grounded: a search is billed when it returns, even if the decision then fails and you retry, so every grounded attempt may be a billed search.

`mode` accepts one value. Anything else returns `400`:

```text theme={null}
{"error":{"code":null,"message":"mode must be \"web_grounding\" when provided","param":"mode","type":"invalid_request_error"}}
```

> [accordion] decide()
>
> ```python factcheck.py (part 2 of 4) theme={null}
> class DecisionsError(Exception):
>     """A Decisions API request that did not return an answer."""
>
>
> @dataclass
> class Decision:
>     probability: float
>     input_tokens: int
>     latency_s: float  # wall time for the claim, retries included
>     attempts: int  # POSTs sent; when grounded, each one may have run a billed search
>     results: list[str] = field(default_factory=list)  # search_results URLs; [] without mode or evidence
>
>
> def is_probability(p: object) -> bool:
>     return isinstance(p, (int, float)) and not isinstance(p, bool) and 0.0 <= p <= 1.0
>
>
> def decide(client: httpx.Client, claim: str, grounded: bool, model: str = MODEL) -> Decision:
>     """POST one claim with one question. `mode` is sent only when grounded."""
>     body: dict[str, object] = {"model": model, "state": claim, "questions": {QUESTION_NAME: QUESTION}}
>     if grounded:
>         body["mode"] = "web_grounding"
>     started = time.perf_counter()
>     for attempt in range(RETRIES + 1):
>         response = client.post(DECISIONS_URL, json=body)
>         if response.status_code == 200:
>             try:
>                 data = response.json()
>                 p, tokens = data["answers"][QUESTION_NAME]["noul"], data["usage"]["input_tokens"]
>                 urls = [r["url"] for r in data.get("search_results") or []]
>             except (ValueError, KeyError, TypeError) as error:
>                 raise DecisionsError(f"200 with an unexpected body: {error!r}") from None
>             if not is_probability(p) or isinstance(tokens, bool) or not isinstance(tokens, int) or tokens < 0:
>                 raise DecisionsError(f"200 with an unexpected body: noul={p!r}, input_tokens={tokens!r}")
>             return Decision(p, tokens, time.perf_counter() - started, attempt + 1, urls)
>         if attempt == RETRIES or not (response.status_code == 429 or response.status_code >= 500):
>             break
>         retry_after = response.headers.get("Retry-After", "")
>         time.sleep(min(int(retry_after) if retry_after.isdigit() else 2**attempt, MAX_WAIT_S))
>     try:
>         message = response.json()["error"]["message"]
>     except (ValueError, KeyError, TypeError):
>         message = response.text.strip()
>     raise DecisionsError(f"{response.status_code}: {message}")
> ```

### Verdicts and the scorecard

`verdict` is the whole policy: at or above `0.80` supported, at or below `0.20` contradicted, otherwise review. Both cutoffs are inclusive.

`evaluate` scores one mode against the labels. A verdict is correct when `supported` meets a true label or `contradicted` meets a false one, wrong when they cross, and `review` is counted on its own. Wrong verdicts split into `missed_failures`, false claims marked supported, and `false_contradictions`, true claims marked contradicted. The Brier score is the mean squared gap between the probability and the label, so a confident wrong answer costs more than an unsure one.

`flips` counts claims whose verdict changed between the two modes. `grounded_attempts` is the number of POSTs sent with `mode` set, an upper bound on billed searches; `mean_search_results` is the evidence count per claim; and cost per 1,000 claims is the mean input tokens at the model's rate plus one search per attempt. It rejects empty input, mismatched lengths, labels that are not booleans, and probabilities outside `[0, 1]`, in the baseline as well as the results.

`accepted` holds a run to three bounds at once: `MAX_MISSED_FAILURES = 0`, `MAX_FALSE_CONTRADICTED_RATE = 0.05`, and `MAX_REVIEW_RATE = 0.25`. Each bound exists because a trivial policy passes the other two: marking everything contradicted has no missed failures and no reviews, and reviewing everything has no wrong verdicts at all. The tests check both of those fail and that the shape of the recorded grounded run passes. Set the bounds from your own tolerance, but keep all three.

> [accordion] verdict(), Summary, accepted(), and evaluate()
>
> ```python factcheck.py (part 3 of 4) theme={null}
> def verdict(p: float) -> str:
>     return "supported" if p >= SUPPORTED_MIN else "contradicted" if p <= CONTRADICTED_MAX else "review"
>
>
> @dataclass
> class Summary:
>     mode: str
>     claims: int
>     correct: int
>     wrong: int
>     review: int
>     missed_failures: int  # false claims marked supported
>     false_contradictions: int  # true claims marked contradicted
>     flips: int | None  # None when no baseline was given
>     brier: float
>     mean_input_tokens: float
>     mean_latency_s: float
>     grounded_attempts: int  # upper bound on billed searches
>     mean_search_results: float
>     cost_per_1000: float
>
>
> def accepted(s: Summary) -> bool:
>     return (s.missed_failures <= MAX_MISSED_FAILURES and s.review <= MAX_REVIEW_RATE * s.claims
>             and s.false_contradictions <= MAX_FALSE_CONTRADICTED_RATE * s.claims)
>
>
> def evaluate(rows: list[dict], results: list[Decision], grounded: bool, model: str, baseline=None) -> Summary:
>     """Score one mode against the labels. `baseline` is the other mode's results, for flips."""
>     n = len(rows)
>     for name, res in [(k, v) for k, v in (("results", results), ("baseline", baseline)) if v is not None]:
>         if n == 0 or len(res) != n:
>             raise ValueError(f"{name}: need one decision per row and at least one row")
>         if not all(is_probability(d.probability) for d in res):
>             raise ValueError(f"{name}: every probability must be a number in [0, 1]")
>     if not all(isinstance(r["label"], bool) for r in rows):
>         raise ValueError("every label must be true or false")
>     labels = [r["label"] for r in rows]
>     verdicts = [verdict(d.probability) for d in results]
>     review = verdicts.count("review")
>     correct = sum(v != "review" and (v == "supported") == label for v, label in zip(verdicts, labels))
>     missed = sum(v == "supported" and not label for v, label in zip(verdicts, labels))
>     false_contra = sum(v == "contradicted" and label for v, label in zip(verdicts, labels))
>     brier = sum((d.probability - float(label)) ** 2 for d, label in zip(results, labels)) / n
>     flips = None if baseline is None else sum(verdict(b.probability) != v for b, v in zip(baseline, verdicts))
>     mean_tokens = sum(d.input_tokens for d in results) / n
>     mean_latency = sum(d.latency_s for d in results) / n
>     attempts = sum(d.attempts for d in results) if grounded else 0
>     cost = 1000 * (mean_tokens * PRICE_PER_M_INPUT[model] / 1e6 + PRICE_PER_SEARCH * attempts / n)
>     return Summary("grounded" if grounded else "ungrounded", n, correct, n - correct - review, review, missed,
>                    false_contra, flips, brier, mean_tokens, mean_latency, attempts,
>                    sum(len(d.results) for d in results) / n, cost)
> ```

### Command line

`load_claims` reads the JSONL and names the line of anything malformed: a missing `claim`, a `label` that is not `true` or `false`, an empty file. `check` runs one mode over the file, `eval` runs both and reports flips. `--pad-before N` puts `N` characters of `FILLER` in front of each claim and `--pad-after N` puts them after it; both may be given. `--model` switches to the flash model, which bills input at half the rate.

Before any request goes out, the script runs `claim_in_query` on every claim and warns when the complete text of any claim is outside its query. The table has one probability and one verdict per mode. `check` prints the `search_results` URLs under each row, so you can see the evidence behind a verdict; `eval` leaves them out of the table and adds `mean search results` to the grounded summary line. The summary is two lines per mode and ends with `acceptance met` or `not met`. Every run writes its rows, results, and summaries to `out/` as JSON.

> [accordion] load\_claims() and main()
>
> ```python factcheck.py (part 4 of 4) theme={null}
> def load_claims(path: str) -> list[dict]:
>     """Read JSONL rows: string `id`, `claim`, `kind` and a bool `label`. Errors name the line."""
>     rows = []
>     for number, line in enumerate(Path(path).read_text(encoding="utf-8").splitlines(), 1):
>         if not line.strip():
>             continue
>         try:
>             row = json.loads(line)
>         except ValueError as error:
>             raise ValueError(f"{path}:{number}: not JSON ({error})") from None
>         strings = isinstance(row, dict) and all(
>             isinstance(row.get(k), str) and row[k].strip() for k in ("id", "claim", "kind"))
>         if not strings or not isinstance(row.get("label"), bool):
>             raise ValueError(f"{path}:{number}: needs string id, claim, kind and a true or false label")
>         rows.append(row)
>     if not rows:
>         raise ValueError(f"{path}: no claims")
>     return rows
>
>
> def main() -> int:
>     parser = argparse.ArgumentParser(description=__doc__)
>     parser.add_argument("command", choices=["check", "eval"])
>     parser.add_argument("claims", help="JSONL with id, claim, label, kind per line")
>     parser.add_argument("--grounded", action="store_true", help="check: send mode=web_grounding")
>     parser.add_argument("--pad-before", type=int, default=0, help="put N chars of filler before each claim")
>     parser.add_argument("--pad-after", type=int, default=0, help="put N chars of filler after each claim")
>     parser.add_argument("--model", default=MODEL, choices=list(PRICE_PER_M_INPUT))
>     args = parser.parse_args()
>     if (args.command == "eval" and args.grounded) or min(args.pad_before, args.pad_after) < 0:
>         parser.error("eval runs both modes (--grounded applies to check), and padding must be 0 or more")
>     try:
>         if not (api_key := os.environ.get("PERPLEXITY_API_KEY")):
>             raise ValueError("set PERPLEXITY_API_KEY first")
>         rows = load_claims(args.claims)
>     except (OSError, ValueError) as error:
>         print(f"cannot start: {error}", file=sys.stderr)
>         return 2
>
>     texts = [pad(r["claim"], args.pad_before, args.pad_after) for r in rows]
>     modes = ["ungrounded", "grounded"]  # eval runs both; check runs one
>     if args.command == "check":
>         modes = ["grounded" if args.grounded else "ungrounded"]
>     in_query = sum(claim_in_query(r["claim"], t, QUESTION) for r, t in zip(rows, texts))  # complete claim text
>     if "grounded" in modes and in_query < len(texts):
>         print(f"warning: {len(texts) - in_query} of {len(texts)} claims are not in their search query; "
>               "the search sees the question and the first bytes of the state.", file=sys.stderr)
>
>     with httpx.Client(headers={"Authorization": f"Bearer {api_key}"}, timeout=30.0) as client:
>         try:
>             results = {m: [decide(client, t, m == "grounded", args.model) for t in texts] for m in modes}
>         except (DecisionsError, httpx.HTTPError) as error:
>             print(f"Decisions API failed: {error}", file=sys.stderr)
>             return 1
>
>     columns = "".join(f"  {'p_' + m:>12}  {'verdict_' + m:<18}" for m in modes)
>     print(f"{'id':<4} {'kind':<7} {'label':<5}{columns}".rstrip())
>     for i, row in enumerate(rows):
>         verdicts = [verdict(results[m][i].probability) for m in modes]
>         cells = "".join(f"  {results[m][i].probability:>12.2f}  {v:<18}" for m, v in zip(modes, verdicts))
>         flip = "  flip" if len(set(verdicts)) > 1 else ""
>         print(f"{row['id']:<4} {row['kind']:<7} {str(row['label']).lower():<5}{cells}{flip}".rstrip())
>         if args.command == "check":  # the evidence behind the verdict, one URL per line
>             print("".join(f"{'':<18}{url}\n" for url in results[modes[0]][i].results), end="")
>     print(f"\nclaim text in the search query: {in_query} of {len(texts)}")
>     pair = {"ungrounded": "grounded", "grounded": "ungrounded"}
>     summaries = [evaluate(rows, results[m], m == "grounded", args.model, results.get(pair[m])) for m in modes]
>     for s in summaries:
>         print(
>             f"{s.mode:<10} correct {s.correct}/{s.claims}  wrong {s.wrong}  review {s.review} "
>             f"({s.review / s.claims:.0%}, bound {MAX_REVIEW_RATE:.0%})  missed failures {s.missed_failures} "
>             f"(bound {MAX_MISSED_FAILURES})  false contradictions {s.false_contradictions} "
>             f"(bound {MAX_FALSE_CONTRADICTED_RATE:.0%})  Brier {s.brier:.3f}"
>             + (f"  flips {s.flips}" if s.flips is not None else "") + "\n"
>             f"{'':<10} mean input tokens {s.mean_input_tokens:.0f}  mean latency {s.mean_latency_s:.2f} s  "
>             f"grounded attempts {s.grounded_attempts}  "
>             + (f"mean search results {s.mean_search_results:.1f}  " if s.mode == "grounded" else "")
>             + f"cost per 1,000 claims ${s.cost_per_1000:.4f}  "
>             f"acceptance {'met' if accepted(s) else 'not met'}"
>         )
>     if "grounded" in results:
>         print("grounded attempts = POSTs sent with mode set, an upper bound on billed searches.")
>
>     OUT_DIR.mkdir(exist_ok=True)
>     stamp = time.strftime("%Y%m%dT%H%M%S")
>     out = OUT_DIR / f"{args.command}-before{args.pad_before}-after{args.pad_after}-{stamp}.json"
>     record = {"model": args.model, "pad_before": args.pad_before, "pad_after": args.pad_after, "rows": rows,
>               "summaries": [asdict(s) for s in summaries]}
>     record["results"] = {m: [asdict(d) for d in r] for m, r in results.items()}
>     out.write_text(json.dumps(record, indent=1), encoding="utf-8")
>     print(f"saved {out}")
>     return 0
>
>
> if __name__ == "__main__":
>     sys.exit(main())
> ```

### The full file

Save this as `factcheck.py`. It is the four parts above in order, separated by two blank lines.

> [accordion] Full code - factcheck.py
>
> ```python factcheck.py theme={null}
> """Fact-check claims against the web with the Decisions API. One request per claim: the claim is the
> `state`, one `noul` question asks whether it is true, and `mode: "web_grounding"` adds a web search."""
>
> import argparse
> import json
> import os
> import sys
> import time
> from dataclasses import asdict, dataclass, field
> from pathlib import Path
>
> import httpx
>
> from query import claim_in_query
>
> DECISIONS_URL = "https://api.perplexity.ai/v1/decisions"
> MODEL = "pplx-decider-latest"  # selects pplx-decider-v1.2
> QUESTION_NAME = "is_true"
> QUESTION = {"type": "noul", "instructions": "Is the claim true?"}
>
> SUPPORTED_MIN = 0.80  # p >= this: supported. Your code applies the cutoffs, not the model.
> CONTRADICTED_MAX = 0.20  # p <= this: contradicted; anything between: review
> MAX_MISSED_FAILURES = 0  # acceptance bounds the eval reports against: false claims marked supported
> MAX_FALSE_CONTRADICTED_RATE = 0.05  # true claims marked contradicted, as a share of claims
> MAX_REVIEW_RATE = 0.25  # share of claims sent to a person
>
> PRICE_PER_M_INPUT = {"pplx-decider-latest": 0.02, "pplx-decider-v1.2": 0.02,  # $ per million input tokens
>                      "pplx-decider-flash-latest": 0.01, "pplx-decider-flash-v1.2": 0.01}
> PRICE_PER_SEARCH = 0.001  # dollars per web search when grounded
> RETRIES = 3  # on 429 and 5xx; waits Retry-After seconds when it is an integer, else 1, 2, 4
> MAX_WAIT_S = 30  # cap on any single wait
> OUT_DIR = Path("out")
>
> FILLER = (  # neutral text for --pad-before and --pad-after; nothing in it relates to any claim
>     "The facilities team confirmed the office move for the second week of the quarter. "
>     "Badge readers on the third floor will be replaced on Monday, and visitor parking "
>     "moves to the lower level. The updated seating chart goes out by email once the "
>     "furniture vendor confirms delivery. Please label boxes with your team name. ")
>
>
> def pad(claim: str, before: int, after: int) -> str:
>     """The claim on its own line, after `before` characters of FILLER and before `after` more."""
>     def filler(n: int) -> list[str]:
>         return [(FILLER * (n // len(FILLER) + 1))[:n]] if n else []
>     return "\n".join(filler(before) + [claim] + filler(after))
>
>
> class DecisionsError(Exception):
>     """A Decisions API request that did not return an answer."""
>
>
> @dataclass
> class Decision:
>     probability: float
>     input_tokens: int
>     latency_s: float  # wall time for the claim, retries included
>     attempts: int  # POSTs sent; when grounded, each one may have run a billed search
>     results: list[str] = field(default_factory=list)  # search_results URLs; [] without mode or evidence
>
>
> def is_probability(p: object) -> bool:
>     return isinstance(p, (int, float)) and not isinstance(p, bool) and 0.0 <= p <= 1.0
>
>
> def decide(client: httpx.Client, claim: str, grounded: bool, model: str = MODEL) -> Decision:
>     """POST one claim with one question. `mode` is sent only when grounded."""
>     body: dict[str, object] = {"model": model, "state": claim, "questions": {QUESTION_NAME: QUESTION}}
>     if grounded:
>         body["mode"] = "web_grounding"
>     started = time.perf_counter()
>     for attempt in range(RETRIES + 1):
>         response = client.post(DECISIONS_URL, json=body)
>         if response.status_code == 200:
>             try:
>                 data = response.json()
>                 p, tokens = data["answers"][QUESTION_NAME]["noul"], data["usage"]["input_tokens"]
>                 urls = [r["url"] for r in data.get("search_results") or []]
>             except (ValueError, KeyError, TypeError) as error:
>                 raise DecisionsError(f"200 with an unexpected body: {error!r}") from None
>             if not is_probability(p) or isinstance(tokens, bool) or not isinstance(tokens, int) or tokens < 0:
>                 raise DecisionsError(f"200 with an unexpected body: noul={p!r}, input_tokens={tokens!r}")
>             return Decision(p, tokens, time.perf_counter() - started, attempt + 1, urls)
>         if attempt == RETRIES or not (response.status_code == 429 or response.status_code >= 500):
>             break
>         retry_after = response.headers.get("Retry-After", "")
>         time.sleep(min(int(retry_after) if retry_after.isdigit() else 2**attempt, MAX_WAIT_S))
>     try:
>         message = response.json()["error"]["message"]
>     except (ValueError, KeyError, TypeError):
>         message = response.text.strip()
>     raise DecisionsError(f"{response.status_code}: {message}")
>
>
> def verdict(p: float) -> str:
>     return "supported" if p >= SUPPORTED_MIN else "contradicted" if p <= CONTRADICTED_MAX else "review"
>
>
> @dataclass
> class Summary:
>     mode: str
>     claims: int
>     correct: int
>     wrong: int
>     review: int
>     missed_failures: int  # false claims marked supported
>     false_contradictions: int  # true claims marked contradicted
>     flips: int | None  # None when no baseline was given
>     brier: float
>     mean_input_tokens: float
>     mean_latency_s: float
>     grounded_attempts: int  # upper bound on billed searches
>     mean_search_results: float
>     cost_per_1000: float
>
>
> def accepted(s: Summary) -> bool:
>     return (s.missed_failures <= MAX_MISSED_FAILURES and s.review <= MAX_REVIEW_RATE * s.claims
>             and s.false_contradictions <= MAX_FALSE_CONTRADICTED_RATE * s.claims)
>
>
> def evaluate(rows: list[dict], results: list[Decision], grounded: bool, model: str, baseline=None) -> Summary:
>     """Score one mode against the labels. `baseline` is the other mode's results, for flips."""
>     n = len(rows)
>     for name, res in [(k, v) for k, v in (("results", results), ("baseline", baseline)) if v is not None]:
>         if n == 0 or len(res) != n:
>             raise ValueError(f"{name}: need one decision per row and at least one row")
>         if not all(is_probability(d.probability) for d in res):
>             raise ValueError(f"{name}: every probability must be a number in [0, 1]")
>     if not all(isinstance(r["label"], bool) for r in rows):
>         raise ValueError("every label must be true or false")
>     labels = [r["label"] for r in rows]
>     verdicts = [verdict(d.probability) for d in results]
>     review = verdicts.count("review")
>     correct = sum(v != "review" and (v == "supported") == label for v, label in zip(verdicts, labels))
>     missed = sum(v == "supported" and not label for v, label in zip(verdicts, labels))
>     false_contra = sum(v == "contradicted" and label for v, label in zip(verdicts, labels))
>     brier = sum((d.probability - float(label)) ** 2 for d, label in zip(results, labels)) / n
>     flips = None if baseline is None else sum(verdict(b.probability) != v for b, v in zip(baseline, verdicts))
>     mean_tokens = sum(d.input_tokens for d in results) / n
>     mean_latency = sum(d.latency_s for d in results) / n
>     attempts = sum(d.attempts for d in results) if grounded else 0
>     cost = 1000 * (mean_tokens * PRICE_PER_M_INPUT[model] / 1e6 + PRICE_PER_SEARCH * attempts / n)
>     return Summary("grounded" if grounded else "ungrounded", n, correct, n - correct - review, review, missed,
>                    false_contra, flips, brier, mean_tokens, mean_latency, attempts,
>                    sum(len(d.results) for d in results) / n, cost)
>
>
> def load_claims(path: str) -> list[dict]:
>     """Read JSONL rows: string `id`, `claim`, `kind` and a bool `label`. Errors name the line."""
>     rows = []
>     for number, line in enumerate(Path(path).read_text(encoding="utf-8").splitlines(), 1):
>         if not line.strip():
>             continue
>         try:
>             row = json.loads(line)
>         except ValueError as error:
>             raise ValueError(f"{path}:{number}: not JSON ({error})") from None
>         strings = isinstance(row, dict) and all(
>             isinstance(row.get(k), str) and row[k].strip() for k in ("id", "claim", "kind"))
>         if not strings or not isinstance(row.get("label"), bool):
>             raise ValueError(f"{path}:{number}: needs string id, claim, kind and a true or false label")
>         rows.append(row)
>     if not rows:
>         raise ValueError(f"{path}: no claims")
>     return rows
>
>
> def main() -> int:
>     parser = argparse.ArgumentParser(description=__doc__)
>     parser.add_argument("command", choices=["check", "eval"])
>     parser.add_argument("claims", help="JSONL with id, claim, label, kind per line")
>     parser.add_argument("--grounded", action="store_true", help="check: send mode=web_grounding")
>     parser.add_argument("--pad-before", type=int, default=0, help="put N chars of filler before each claim")
>     parser.add_argument("--pad-after", type=int, default=0, help="put N chars of filler after each claim")
>     parser.add_argument("--model", default=MODEL, choices=list(PRICE_PER_M_INPUT))
>     args = parser.parse_args()
>     if (args.command == "eval" and args.grounded) or min(args.pad_before, args.pad_after) < 0:
>         parser.error("eval runs both modes (--grounded applies to check), and padding must be 0 or more")
>     try:
>         if not (api_key := os.environ.get("PERPLEXITY_API_KEY")):
>             raise ValueError("set PERPLEXITY_API_KEY first")
>         rows = load_claims(args.claims)
>     except (OSError, ValueError) as error:
>         print(f"cannot start: {error}", file=sys.stderr)
>         return 2
>
>     texts = [pad(r["claim"], args.pad_before, args.pad_after) for r in rows]
>     modes = ["ungrounded", "grounded"]  # eval runs both; check runs one
>     if args.command == "check":
>         modes = ["grounded" if args.grounded else "ungrounded"]
>     in_query = sum(claim_in_query(r["claim"], t, QUESTION) for r, t in zip(rows, texts))  # complete claim text
>     if "grounded" in modes and in_query < len(texts):
>         print(f"warning: {len(texts) - in_query} of {len(texts)} claims are not in their search query; "
>               "the search sees the question and the first bytes of the state.", file=sys.stderr)
>
>     with httpx.Client(headers={"Authorization": f"Bearer {api_key}"}, timeout=30.0) as client:
>         try:
>             results = {m: [decide(client, t, m == "grounded", args.model) for t in texts] for m in modes}
>         except (DecisionsError, httpx.HTTPError) as error:
>             print(f"Decisions API failed: {error}", file=sys.stderr)
>             return 1
>
>     columns = "".join(f"  {'p_' + m:>12}  {'verdict_' + m:<18}" for m in modes)
>     print(f"{'id':<4} {'kind':<7} {'label':<5}{columns}".rstrip())
>     for i, row in enumerate(rows):
>         verdicts = [verdict(results[m][i].probability) for m in modes]
>         cells = "".join(f"  {results[m][i].probability:>12.2f}  {v:<18}" for m, v in zip(modes, verdicts))
>         flip = "  flip" if len(set(verdicts)) > 1 else ""
>         print(f"{row['id']:<4} {row['kind']:<7} {str(row['label']).lower():<5}{cells}{flip}".rstrip())
>         if args.command == "check":  # the evidence behind the verdict, one URL per line
>             print("".join(f"{'':<18}{url}\n" for url in results[modes[0]][i].results), end="")
>     print(f"\nclaim text in the search query: {in_query} of {len(texts)}")
>     pair = {"ungrounded": "grounded", "grounded": "ungrounded"}
>     summaries = [evaluate(rows, results[m], m == "grounded", args.model, results.get(pair[m])) for m in modes]
>     for s in summaries:
>         print(
>             f"{s.mode:<10} correct {s.correct}/{s.claims}  wrong {s.wrong}  review {s.review} "
>             f"({s.review / s.claims:.0%}, bound {MAX_REVIEW_RATE:.0%})  missed failures {s.missed_failures} "
>             f"(bound {MAX_MISSED_FAILURES})  false contradictions {s.false_contradictions} "
>             f"(bound {MAX_FALSE_CONTRADICTED_RATE:.0%})  Brier {s.brier:.3f}"
>             + (f"  flips {s.flips}" if s.flips is not None else "") + "\n"
>             f"{'':<10} mean input tokens {s.mean_input_tokens:.0f}  mean latency {s.mean_latency_s:.2f} s  "
>             f"grounded attempts {s.grounded_attempts}  "
>             + (f"mean search results {s.mean_search_results:.1f}  " if s.mode == "grounded" else "")
>             + f"cost per 1,000 claims ${s.cost_per_1000:.4f}  "
>             f"acceptance {'met' if accepted(s) else 'not met'}"
>         )
>     if "grounded" in results:
>         print("grounded attempts = POSTs sent with mode set, an upper bound on billed searches.")
>
>     OUT_DIR.mkdir(exist_ok=True)
>     stamp = time.strftime("%Y%m%dT%H%M%S")
>     out = OUT_DIR / f"{args.command}-before{args.pad_before}-after{args.pad_after}-{stamp}.json"
>     record = {"model": args.model, "pad_before": args.pad_before, "pad_after": args.pad_after, "rows": rows,
>               "summaries": [asdict(s) for s in summaries]}
>     record["results"] = {m: [asdict(d) for d in r] for m, r in results.items()}
>     out.write_text(json.dumps(record, indent=1), encoding="utf-8")
>     print(f"saved {out}")
>     return 0
>
>
> if __name__ == "__main__":
>     sys.exit(main())
> ```

## Run it

Save `test_factcheck.py` and run the tests first. They need no key and no network: `httpx.MockTransport` stands in for the API. They cover the request body and `mode`, the error and retry paths, `search_results` in the `Decision` and the record, the scorecard and its bounds, and the claims file and command line with a fake `decide`.

> [accordion] test\_factcheck.py
>
> ```python test_factcheck.py theme={null}
> """Offline checks for factcheck.py. No network: httpx gets a mock transport. Run: python -m pytest -q"""
>
> import json
> import sys
> from pathlib import Path
>
> import httpx
> import pytest
>
> import factcheck as fc
>
> BAD_MODE_BODY = {"error": {"code": None, "message": 'mode must be "web_grounding" when provided',
>                            "param": "mode", "type": "invalid_request_error"}}
> CLAIMS_FILE = Path(__file__).with_name("claims.jsonl")
> ROWS = [{"id": i, "claim": c, "label": label, "kind": k} for i, c, label, k in [
>     ("a", "true one", True, "stable"), ("b", "false one", False, "stable"),
>     ("c", "true two", True, "recent"), ("d", "false two", False, "recent")]]
> GROUNDED = [fc.Decision(p, t, 0.4, 1, ["https://e.com/"] * u) for p, t, u in
>             [(0.9, 100, 5), (0.1, 200, 0), (0.5, 300, 0), (0.85, 400, 0)]]
> UNGROUNDED = [fc.Decision(0.5, 50, 0.3, 1) for _ in ROWS]
>
>
> def ok_response(p: float, tokens: int, urls: list[str] | None = None) -> dict:
>     body = {"model": fc.MODEL, "answers": {fc.QUESTION_NAME: {"type": "noul", "noul": p}},
>             "usage": {"input_tokens": tokens, "output_tokens": 1}}
>     return body if urls is None else {**body, "search_results": [{"url": u, "title": "t"} for u in urls]}
>
>
> def client_with(responses: list[httpx.Response], seen: list[dict]) -> httpx.Client:
>     def handle(request: httpx.Request) -> httpx.Response:  # one canned response per request; bodies recorded
>         seen.append(json.loads(request.content))
>         return responses.pop(0)
>     return httpx.Client(transport=httpx.MockTransport(handle))
>
>
> @pytest.mark.parametrize(("grounded", "model"), [(False, fc.MODEL), (True, "pplx-decider-flash-latest")])
> def test_request_body_sends_mode_only_when_grounded(grounded: bool, model: str) -> None:
>     seen: list[dict] = []
>     urls = ["https://example.com/rust"] if grounded else None
>     with client_with([httpx.Response(200, json=ok_response(0.19, 108, urls))], seen) as client:
>         d = fc.decide(client, "Rust 1.0 was released in May 2015.", grounded, model)
>     expected = {"model": model, "state": "Rust 1.0 was released in May 2015.",
>                 "questions": {fc.QUESTION_NAME: fc.QUESTION}}
>     assert seen == [{**expected, "mode": "web_grounding"} if grounded else expected]
>     assert (d.probability, d.input_tokens, d.attempts, d.results) == (0.19, 108, 1, urls or []) and d.latency_s >= 0
>
>
> @pytest.mark.parametrize(("response", "message"), [
>     (httpx.Response(400, json=BAD_MODE_BODY), '400: mode must be "web_grounding" when provided'),
>     (httpx.Response(401, text="Unauthorized"), "401: Unauthorized"),
>     (httpx.Response(200, json={"answers": {}}), "200 with an unexpected body"),
>     (httpx.Response(200, text="<html>"), "200 with an unexpected body"),
> ] + [(httpx.Response(200, json=ok_response(p, tokens)), "200 with an unexpected body")
>      for p, tokens in [(True, 100), (None, 100), ("0.9", 100), (1.5, 100), (0.9, -1)]]
>   + [(httpx.Response(200, json={**ok_response(0.9, 100), "search_results": [{"title": "no url"}]}),
>       "200 with an unexpected body")])
> def test_bad_responses_raise_without_a_retry(monkeypatch, response: httpx.Response, message: str) -> None:
>     monkeypatch.setattr(fc.time, "sleep", lambda _: pytest.fail("must not sleep"))
>     seen: list[dict] = []
>     with client_with([response], seen) as client:
>         with pytest.raises(fc.DecisionsError) as info:
>             fc.decide(client, "claim", grounded=True)
>     assert str(info.value).startswith(message) and len(seen) == 1
>
>
> @pytest.mark.parametrize(("retry_after", "wait"), [
>     ("3", 3), ("Wed, 21 Oct 2026 07:28:00 GMT", 1), ("-5", 1), ("2.5", 1), ("600", fc.MAX_WAIT_S)])
> def test_retry_waits_then_counts_two_attempts_and_searches(monkeypatch, retry_after: str, wait: float) -> None:
>     waits: list[float] = []
>     monkeypatch.setattr(fc.time, "sleep", waits.append)
>     responses = [httpx.Response(429, headers={"Retry-After": retry_after}, json={"error": {"message": "slow"}}),
>                  httpx.Response(200, json=ok_response(0.5, 100))]
>     with client_with(responses, []) as client:
>         d = fc.decide(client, "claim", grounded=True)
>     assert (d.probability, d.attempts, waits) == (0.5, 2, [wait])
>     assert fc.evaluate(ROWS[:1], [d], grounded=True, model=fc.MODEL).grounded_attempts == 2
>
>
> def test_gives_up_after_retries_on_5xx(monkeypatch) -> None:
>     waits: list[float] = []
>     monkeypatch.setattr(fc.time, "sleep", waits.append)
>     seen: list[dict] = []
>     with client_with([httpx.Response(503, text="upstream") for _ in range(fc.RETRIES + 1)], seen) as client:
>         with pytest.raises(fc.DecisionsError, match="503: upstream"):
>             fc.decide(client, "claim", grounded=False)
>     assert (len(seen), waits) == (fc.RETRIES + 1, [1, 2, 4])  # exponential backoff without Retry-After
>
>
> @pytest.mark.parametrize(("p", "expected"), [
>     (1.0, "supported"), (0.80, "supported"), (0.7999, "review"), (0.2001, "review"), (0.20, "contradicted")])
> def test_verdict_cutoffs_are_inclusive(p: float, expected: str) -> None:
>     assert fc.verdict(p) == expected
>
>
> def test_evaluate_scorecard() -> None:
>     s = fc.evaluate(ROWS, GROUNDED, grounded=True, model=fc.MODEL, baseline=UNGROUNDED)
>     assert (s.correct, s.wrong, s.review) == (2, 1, 1)
>     assert (s.missed_failures, s.false_contradictions) == (1, 0)  # d: false, 0.85 -> supported
>     assert s.flips == 3  # a, b, d changed verdict; c stayed in review
>     assert s.brier == pytest.approx((0.01 + 0.01 + 0.25 + 0.7225) / 4)
>
>
> def test_evaluate_cost_adds_one_search_per_grounded_attempt() -> None:
>     grounded = fc.evaluate(ROWS, GROUNDED, grounded=True, model=fc.MODEL)
>     ungrounded = fc.evaluate(ROWS, UNGROUNDED, grounded=False, model=fc.MODEL)
>     flash = fc.evaluate(ROWS, UNGROUNDED, grounded=False, model="pplx-decider-flash-latest")
>     assert (grounded.mean_input_tokens, grounded.mean_latency_s) == (250, pytest.approx(0.4))
>     assert (grounded.grounded_attempts, ungrounded.grounded_attempts, ungrounded.flips) == (4, 0, None)
>     assert (grounded.mean_search_results, ungrounded.mean_search_results) == (1.25, 0)
>     # 1,000 claims x 250 tokens x $0.02/M = $0.005, plus 1,000 searches x $0.001 = $1.00
>     assert grounded.cost_per_1000 == pytest.approx(1.005)
>     assert ungrounded.cost_per_1000 == pytest.approx(1000 * 50 * 0.02 / 1e6)
>     assert flash.cost_per_1000 == pytest.approx(1000 * 50 * 0.01 / 1e6)
>
>
> @pytest.mark.parametrize(("rows", "results", "baseline"), [
>     ([], [], None),
>     (ROWS, GROUNDED[:3], None),
>     ([{**ROWS[0], "label": "true"}], GROUNDED[:1], None),
>     (ROWS[:1], [fc.Decision(1.5, 100, 0.1, 1)], None),
>     (ROWS[:1], [fc.Decision(float("nan"), 100, 0.1, 1)], None),
>     (ROWS, GROUNDED, UNGROUNDED[:3]),
>     (ROWS[:1], GROUNDED[:1], [fc.Decision(-0.1, 100, 0.1, 1)]),
> ])
> def test_evaluate_rejects_bad_input(rows: list[dict], results: list[fc.Decision], baseline) -> None:
>     with pytest.raises(ValueError):
>         fc.evaluate(rows, results, grounded=False, model=fc.MODEL, baseline=baseline)
>
>
> @pytest.mark.parametrize(("true_p", "false_p", "counts", "passes"), [
>     (0.0, 0.0, (0, 13, 0), False),  # contradict everything
>     (0.5, 0.5, (0, 0, 24), False),  # review everything
>     (0.98, 0.0, (0, 0, 0), True)])  # the recorded grounded run's shape
> def test_acceptance_needs_all_three_bounds(true_p: float, false_p: float, counts: tuple, passes: bool) -> None:
>     rows = fc.load_claims(str(CLAIMS_FILE))
>     results = [fc.Decision(true_p if r["label"] else false_p, 1000, 0.4, 1) for r in rows]
>     s = fc.evaluate(rows, results, grounded=True, model=fc.MODEL)
>     assert (s.missed_failures, s.false_contradictions, s.review) == counts
>     assert fc.accepted(s) is passes
>
>
> @pytest.mark.parametrize(("false_contradictions", "reviews", "passes"), [
>     (1, 0, True), (2, 0, False),  # 1 of 20 is the 5% bound exactly; 2 is over it
>     (0, 5, True), (0, 6, False),  # 5 of 20 is the 25% review bound exactly
> ])
> def test_acceptance_bounds_are_inclusive(false_contradictions: int, reviews: int, passes: bool) -> None:
>     rows = [{"id": f"t{i}", "claim": "c", "label": True, "kind": "stable"} for i in range(20)]
>     ps = [0.0] * false_contradictions + [0.5] * reviews
>     ps += [1.0] * (20 - len(ps))
>     s = fc.evaluate(rows, [fc.Decision(p, 100, 0.1, 1) for p in ps], grounded=False, model=fc.MODEL)
>     assert (s.false_contradictions, s.review) == (false_contradictions, reviews)
>     assert fc.accepted(s) is passes
>
>
> @pytest.mark.parametrize(("content", "where"), [
>     ("", "no claims"),
>     ('{"id": "a", "claim": "ok", "label": true, "kind": "stable"}\n'
>      '{"id": "b", "label": true, "kind": "stable"}\n', ":2:"),
>     ('{"id": "a", "claim": "ok", "label": "true", "kind": "stable"}\n', ":1:"),
>     ("not json\n", ":1: not JSON"),
> ])
> def test_load_claims_names_the_bad_line(tmp_path: Path, content: str, where: str) -> None:
>     (tmp_path / "claims.jsonl").write_text(content, encoding="utf-8")
>     with pytest.raises(ValueError, match=where):
>         fc.load_claims(str(tmp_path / "claims.jsonl"))
>
>
> def test_pad_puts_filler_on_its_own_lines() -> None:
>     claim = "Python 3.12 was released in October 2023."
>     assert fc.pad(claim, 0, 0) == claim
>     assert fc.pad(claim, 600, 0) == fc.pad(claim, 600, 600).rsplit("\n", 1)[0]
>     assert len(fc.pad(claim, 600, 0)) == len(claim) + 601
>     assert fc.pad(claim, 0, 5).splitlines() == [claim, "The f"] and fc.pad(claim, 3, 0).splitlines() == ["The", claim]
>
>
> @pytest.fixture
> def cli(monkeypatch, tmp_path: Path):
>     monkeypatch.chdir(tmp_path)
>     monkeypatch.setenv("PERPLEXITY_API_KEY", "k")
>
>     def run(*args: str) -> int:
>         monkeypatch.setattr(sys, "argv", ["factcheck.py", *args])
>         return fc.main()
>     return run
>
>
> def test_cli_stops_before_any_request_on_bad_input(cli, monkeypatch, tmp_path: Path, capsys) -> None:
>     monkeypatch.setattr(fc, "decide", lambda *a, **k: pytest.fail("must not call the API"))
>     with pytest.raises(SystemExit):
>         cli("eval", str(CLAIMS_FILE), "--grounded")
>     with pytest.raises(SystemExit):
>         cli("check", str(CLAIMS_FILE), "--pad-after", "-1")
>     (tmp_path / "bad.jsonl").write_text('{"id": "a"}\n', encoding="utf-8")
>     assert cli("check", str(tmp_path / "bad.jsonl")) == 2 and "bad.jsonl:1:" in capsys.readouterr().err
>     monkeypatch.delenv("PERPLEXITY_API_KEY")
>     assert cli("check", str(CLAIMS_FILE)) == 2 and "PERPLEXITY_API_KEY" in capsys.readouterr().err
>
>
> def test_cli_check_and_eval_runs(cli, monkeypatch, tmp_path: Path, capsys) -> None:
>     rows = fc.load_claims(str(CLAIMS_FILE))
>     by_text = {t: r for r in rows for t in (r["claim"], fc.pad(r["claim"], 600, 0))}
>
>     def fake_decide(client, text, grounded, model=fc.MODEL) -> fc.Decision:
>         row = by_text[text]
>         if not grounded and row["id"] == "s08":  # a false claim marked supported
>             return fc.Decision(0.85, 200, 0.2, 1)
>         urls = [f"https://example.com/{row['id']}"] if grounded else []
>         return fc.Decision(0.98 if row["label"] else 0.0, 1800 if grounded else 200, 0.3, 1, urls)
>
>     monkeypatch.setattr(fc, "decide", fake_decide)
>     assert cli("check", str(CLAIMS_FILE), "--grounded") == 0
>     out = capsys.readouterr().out
>     assert "p_grounded" in out and "p_ungrounded" not in out and "grounded attempts 24" in out
>     assert "claim text in the search query: 24 of 24" in out and "acceptance met" in out
>     assert "mean search results 1.0" in out and out.count("https://example.com/") == 24  # one URL line per claim
>     assert cli("eval", str(CLAIMS_FILE), "--pad-before", "600") == 0  # the warning fires, ungrounded fails the gate
>     captured = capsys.readouterr()
>     assert "warning: 24 of 24 claims are not in their search query" in captured.err
>     assert "claim text in the search query: 0 of 24" in captured.out and "https://example.com/" not in captured.out
>     assert "missed failures 1 (bound 0)" in captured.out and "acceptance not met" in captured.out
>     record = json.loads(next((tmp_path / "out").glob("eval-before600-after0-*.json")).read_text(encoding="utf-8"))
>     assert [s["mode"] for s in record["summaries"]] == ["ungrounded", "grounded"]
>     first = record["results"]["grounded"][0]
>     assert (first["attempts"], first["results"]) == (1, ["https://example.com/s01"])
>
>     def failing_decide(client, text, grounded, model=fc.MODEL) -> fc.Decision:
>         raise fc.DecisionsError("503: upstream")
>
>     monkeypatch.setattr(fc, "decide", failing_decide)
>     assert cli("check", str(CLAIMS_FILE)) == 1 and "Decisions API failed: 503" in capsys.readouterr().err
> ```

> [accordion] Run the tests
>
> ```bash theme={null}
> python -m pytest -q
> ```
>
> ```text theme={null}
> ........................................................................ [ 86%]
> ...........                                                              [100%]
> 83 passed in 0.19s
> ```

Then check the claims with grounding on. `check` prints one probability and one verdict per claim with the URLs the model saw under it, the query count, the summary for that mode, and the path of the JSON it saved. Every output on this page was recorded on 2026-11-04 with `pplx-decider-latest`.

> [accordion] Check the claims
>
> ```bash theme={null}
> python factcheck.py check claims.jsonl --grounded
> ```
>
> ```text theme={null}
> id   kind    label    p_grounded  verdict_grounded
> s01  stable  true           1.00  supported
>                   https://docs.python.org/3/whatsnew/3.12.html
>                   https://lenz.io/c/python-3-12-release-october-2-2023-4e83fd9d
>                   https://www.python.org/downloads/release/python-3120/
>                   https://www.python.org/doc/versions/
>                   https://docs.python.org/3.12/whatsnew/changelog.html
> s02  stable  true           1.00  supported
>                   https://safeguard.sh/resources/blog/rust-release-history-from-1-0-to-the-2024-edition-six-week-trains-editions-governance-and-security-response
>                   https://www.zdnet.com/article/rust-turns-10-how-a-broken-elevator-changed-software-forever/
>                   https://en.wikipedia.org/wiki/Rust_(programming_language)
>                   https://blog.rust-lang.org/2015/05/15/Rust-1.0/
>                   https://doc.rust-lang.org/edition-guide/rust-2015/index.html
> s03  stable  true           1.00  supported
>                   https://wwf.org.au/what-we-do/oceans/great-barrier-reef
>                   https://oceanservice.noaa.gov/facts/gbrlargeststructure.html
>                   https://en.wikipedia.org/wiki/Great_Barrier_Reef
>                   https://www.museum.qld.gov.au/learn-and-discover/queenslands-environments-and-biodiversity/the-great-barrier-reef/
>                   https://www.gbrmpa.gov.au/learn/fascinating-facts-about-great-barrier-reef
> s04  stable  true           1.00  supported
>                   https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status
>                   https://developers.cloudflare.com/support/troubleshooting/http-status-codes/4xx-client-error/error-404/
>                   https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/404
>                   https://www.context.dev/glossary/http-errors/404
>                   https://404meanings.com/404-meaning/
> s05  stable  false          0.00  contradicted
>                   https://en.wikipedia.org/wiki/History_of_Linux
>                   https://www.redhat.com/en/blog/celebrating-30-years-linux-kernel-and-gplv2
>                   https://www.linuxfoundation.org/blog/blog/anniversary-of-first-linux-kernel-release-a-look-at-collaborative-value
>                   https://opensource.com/life/15/8/happy-24th-birthday-linux-kernel
>                   https://lwn.net/1999/features/1998timeline/
> s06  stable  false          0.00  contradicted
>                   https://www.britannica.com/story/7-or-8-summits-the-worlds-highest-mountains-by-continent
>                   https://en.tempo.co/read/2118799/7-highest-mountains-in-the-world-one-from-each-continent
>                   https://www.il.tzembassy.go.tz/resources/view/12-interesting-facts-about-mount-kilimanjaro
>                   https://www.nepalhikingteam.com/seven-summits-highest-peaks-by-continent
>                   https://thebucketlistcompany.co.uk/8-mount-kilimanjaro-facts-to-wow-your-fellow-climbers/
> s07  stable  false          0.00  contradicted
>                   https://en.wikipedia.org/wiki/Guido_van_Rossum
>                   https://www.devclass.com/development/2025/04/11/20-years-of-git-never-a-big-thing-for-me-says-inventor-linus-torvalds/1620325
>                   https://about.gitlab.com/blog/celebrating-gits-20th-anniversary-with-creator-linus-torvalds/
>                   https://www.welcometothejungle.com/en/articles/btc-history-git
>                   https://www.geeksacademy.it/en-en/article-83/who-is-the-father-of-python-one-of-the-most-used-programming-languages-in-the-world
> s08  stable  false          0.00  contradicted
>                   https://www.postgresql.org/about/licence/?lang=en
>                   https://www.postgresql.org/about/press/faq/
>                   https://wiki.postgresql.org/wiki/FAQ
>                   https://en.wikipedia.org/wiki/PostgreSQL
>                   https://news.ycombinator.com/item?id=25932194
> r01  recent  true           1.00  supported
>                   https://www.nytimes.com/live/2026/10/09/world/nobel-peace-prize-winner-2026/heres-the-latest
>                   https://www.cnn.com/2026/10/09/world/live-news/nobel-peace-prize-2026-winner-intl
>                   https://www.bbc.com/news/articles/cm9wz5kng0x1o
>                   https://news.un.org/en/story/2026/10/1168558
>                   https://www.nobelprize.org/educational/nobel-prize-lessons/text-for-students-peace-2026/
> r02  recent  true           1.00  supported
>                   https://physics.berkeley.edu/news/berkeley-helped-make-icecube-success%E2%80%94and-earn-its-leader-2026-nobel-prize
>                   https://www.reuters.com/science/francis-halzen-wins-2026-nobel-prize-physics-2026-10-06/
>                   https://www.opb.org/article/2026/10/06/francis-halzen-wins-nobel-prize-in-physics-for-work-on-high-energy-neutrinos/
>                   https://www.nature.com/articles/d41586-026-03190-0?error=cookies_not_supported&code=32e41dd9-8c59-473b-a396-d743165705c8
>                   https://www.bbc.com/news/articles/cq203mymlvkeo
> r03  recent  true           1.00  supported
>                   https://www.reuters.com/5f374b32b07b/lifestyle/canadian-writer-anne-carson-wins-2026-nobel-prize-literature-2026-10-08/
>                   https://www.npr.org/2026/10/08/nx-s1-5984350/nobel-prize-in-literature-anne-carson
>                   https://ca.rollingstone.com/culture/canadian-anne-carson-wins-nobel-prize/
>                   https://www.cnn.com/2026/10/08/style/nobel-prize-literature-anne-carson-winner-2026-intl
>                   https://amp.dw.com/en/nobel-prize-in-literature-goes-to-canadas-anne-carson/a-79594136
> r04  recent  true           1.00  supported
>                   https://link.springer.com/article/10.1007/s11224-026-02931-1
>                   https://www.cnrs.fr/en/press/2026-nobel-prize-chemistry-awarded-henri-kagan-his-work-asymmetric-catalysis
>                   https://www.nobelprize.org/prizes/chemistry/2026/popular-information/
>                   https://www.nobelprize.org/prizes/chemistry/2026/summary/
>                   https://www.cnn.com/2026/10/07/science/nobel-prize-chemistry-2026-henri-kagan-kenso-soai-winner-intl
> r05  recent  true           1.00  supported
>                   https://www.cnn.com/2026/10/28/sport/live-news/game-5-world-series
>                   https://www.mlb.com/brewers/news/brewers-beat-rays-game-5-win-2026-world-series
>                   https://www.mlb.com/brewers/news/brewers-win-world-series-2026
>                   https://www.espn.com/mlb/story/_/id/50294394/world-series-brewers-champions-franchise-first-title
>                   https://www.usatoday.com/story/sports/mlb/2026/10/29/brewers-first-world-series-title-mlb-teams-never-won-world-series/92490734007/
> r06  recent  true           1.00  supported
>                   https://blog.python.org/2026/10/python-3150-final-is-here/
>                   https://docs.python.org/3/whatsnew/changelog.html
>                   https://realpython.com/python-news-november-2026/
>                   https://xt.pt/software/python-3-15-0-ships-eight-days-late-and-the-lazy-import-fixes-are-still-coming
>                   https://realpython.com/python-news-october-2026/
> r07  recent  true           1.00  supported
>                   https://global.espn.com/football/story/_/id/50236983/harry-kane-2026-ballon-dor-kylian-mbappe-lamine-yamal
>                   https://www.nytimes.com/athletic/7710586/2026/10/26/harry-kane-ballon-dor-winner-2026/
>                   https://www.premierleague.com/en/news/4749936/harry-kane-wins-ballon-dor-as-premier-league-stars-learn-rankings
>                   https://www.skysports.com/football/news/13601934/harry-kane-wins-2026-ballon-dor-bayern-munich-striker-becomes-first-english-winner-of-award-in-25-years
>                   https://www.uefa.com/uefachampionsleague/news/02aa-21ec47378e70-b29defb43769-1000--harry-kane-wins-2026-men-s-ballon-d-or/
> r08  recent  true           1.00  supported
>                   https://www.federalreserve.gov/newsevents/pressreleases/monetary20261028a1.htm
>                   https://www.federalreserve.gov/monetarypolicy/files/monetary20261028a1.pdf
>                   https://www.federalreserve.gov/newsevents/pressreleases/monetary20261028a.htm?
>                   https://www.cnbc.com/2026/10/28/fed-meeting-today-live-updates.html
>                   https://finance.yahoo.com/economy/policy/articles/fed-holds-rates-mortgage-relief-185653601.html
> r09  recent  false          0.00  contradicted
>                   https://blog.pureinventionbook.com/p/a-wild-prize-chase
>                   https://indianexpress.com/article/books-and-literature/nobel-prize-in-literature-2026-live-updates-classicist-poet-or-chinas-dream-weaver-anne-carson-and-can-xue-go-head-to-head-on-bookies-lists-10912371/
>                   https://note.com/tsuyoshin38/n/n3e74313c9373?hl=en
>                   https://www.britannica.com/biography/Haruki-Murakami
>                   https://believermagazine.substack.com/p/les-nobelisables-haruki-murakami
> r10  recent  false          0.00  contradicted
>                   https://www.nbcnews.com/sports/mlb/milwaukee-brewers-tampa-bay-rays-world-series-game-5-result-rcna605974
>                   https://www.bbc.com/sport/baseball/articles/c5x2v991l0lvo
>                   https://sports.yahoo.com/mlb/live/brewers-rays-world-series-game-5-box-score-live-updates-lineups-starting-pitchers-highlights-where-to-watch-230000010.html
>                   https://sports.yahoo.com/mlb/article/world-series-2026-for-the-rays-being-the-best-team-in-the-american-league-simply-wasnt-enough-074822560.html
>                   https://www.philstar.com/campus/sports/2026/10/30/2566690/long-wait-finally-over
> r11  recent  false          0.00  contradicted
>                   https://www.espn.com/soccer/story/_/id/50295834/why-real-madrid-kylian-mbappe-barcelona-lamine-yamal-lost-ballon-dor-harry-kane
>                   https://global.espn.com/football/story/_/id/50289230/ballon-dor-2026-winners-blunder-harry-kane-lamine-yamal-kylian-mbappe
>                   https://www.espn.com/soccer/story/_/id/50289230/ballon-dor-2026-winners-blunder-harry-kane-lamine-yamal-kylian-mbappe
>                   https://global.espn.com/football/story/_/id/50236983/harry-kane-2026-ballon-dor-kylian-mbappe-lamine-yamal
>                   https://www.cbssports.com/soccer/news/harry-kane-alexia-putellas-win-2026-ballon-dor-awards/live/
> r12  recent  false          0.00  contradicted
>                   https://www.federalreserve.gov/newsevents/pressreleases/monetary20261028a1.htm
>                   https://www.federalreserve.gov/monetarypolicy/files/monetary20261028a1.pdf
>                   https://www.jpmorgan.com/insights/markets-and-economy/economy/kevin-warsh-october-2026-federal-reserve-holds-rates-vows-to-deliver-price-stability
>                   https://www.cnbc.com/2026/10/28/fed-meeting-today-live-updates.html
>                   https://www.courant.com/2026/10/28/fed-meeting-interest-rates/
> r13  recent  false          0.00  contradicted
>                   https://www.apple.com/newsroom/2026/09/apple-debuts-iphone-18-pro-and-iphone-18-pro-max/
>                   https://www.macworld.com/article/672537/when-is-apples-next-event.html
>                   https://www.theapplepost.com/2026/09/10/72070/apple-event-rumors-vs-reality-what-the-leaks-got-right-what-they-got-wrong-and-the-biggest-surprises/
>                   https://www.bytepreview.com/en/articulos/no-confirmed-phone-launches-in-october-2026-what-is-really-happening
>                   https://www.apple.com/newsroom/2026/09/the-latest-iphone-apple-watch-and-airpods-lineups-arrive-in-stores-worldwide/
> r14  recent  false          0.00  contradicted
>                   https://www.nobelprize.org/prizes/physics/2026/prize-announcement/
>                   https://www.nobelprize.org/prizes/physics/2026/summary/
>                   https://www.nobelprize.org/prizes/physics/2026/advanced-information/
>                   https://spectator.com/article/why-im-sceptical-about-a-superconductor-breakthrough/
>                   https://www.nobelprize.org/physics-prize-2/
> r15  recent  false          0.00  contradicted
>                   https://www.nobelprize.org/prizes/economic-sciences/all-prizes-in-economic-sciences/
>                   https://www.nobelprize.org/prizes/economic-sciences/
>                   https://www.nobelprize.org/prizes/lists/all-prizes-in-economic-sciences/all/
>                   https://www.nber.org/nber-news/nobel-laureates
>                   https://www.nobelprize.org/prizes/economic-sciences/2024/summary/summary/
> r16  recent  true           1.00  supported
>                   https://www.cbc.ca/news/world/nobel-prize-sweden-medicine-physiology-9.7366536
>                   https://www.nobelprize.org/prizes/medicine/2026/popular-information/
>                   https://www.nbcnews.com/world/europe/nobel-prize-medicine-2026-karl-deisseroth-peter-hegemann-georg-nagel-rcna601564
>                   https://www.nobelprize.org/prizes/medicine/2026/summary/
>                   https://www.smithsonianmag.com/smart-news/trio-of-scientists-wins-medicine-nobel-for-a-method-to-turn-nerve-cells-on-and-off-with-light-helping-to-unravel-the-brains-mysteries-180989618/
>
> claim text in the search query: 24 of 24
> grounded   correct 24/24  wrong 0  review 0 (0%, bound 25%)  missed failures 0 (bound 0)  false contradictions 0 (bound 5%)  Brier 0.000
>            mean input tokens 1908  mean latency 0.50 s  grounded attempts 24  mean search results 5.0  cost per 1,000 claims $1.0382  acceptance met
> grounded attempts = POSTs sent with mode set, an upper bound on billed searches.
> saved out/check-before0-after0-20261104T174406.json
> ```

Drop `--grounded` to get the probability without a search.

## Measure it

Run `eval` on the same file. It sends every claim twice, without `mode` and with it, and lines the two up; the rest of this page reads from it.

> [accordion] Run both modes
>
> ```bash theme={null}
> python factcheck.py eval claims.jsonl
> ```

> [accordion] Observed output
>
> ```text theme={null}
> id   kind    label  p_ungrounded  verdict_ungrounded    p_grounded  verdict_grounded
> s01  stable  true           1.00  supported                   1.00  supported
> s02  stable  true           1.00  supported                   1.00  supported
> s03  stable  true           1.00  supported                   1.00  supported
> s04  stable  true           1.00  supported                   1.00  supported
> s05  stable  false          0.04  contradicted                0.00  contradicted
> s06  stable  false          0.00  contradicted                0.00  contradicted
> s07  stable  false          0.00  contradicted                0.00  contradicted
> s08  stable  false          0.15  contradicted                0.00  contradicted
> r01  recent  true           0.01  contradicted                1.00  supported           flip
> r02  recent  true           0.04  contradicted                1.00  supported           flip
> r03  recent  true           0.04  contradicted                1.00  supported           flip
> r04  recent  true           0.04  contradicted                1.00  supported           flip
> r05  recent  true           0.10  contradicted                1.00  supported           flip
> r06  recent  true           0.07  contradicted                1.00  supported           flip
> r07  recent  true           0.13  contradicted                1.00  supported           flip
> r08  recent  true           0.24  review                      1.00  supported           flip
> r09  recent  false          0.02  contradicted                0.00  contradicted
> r10  recent  false          0.04  contradicted                0.00  contradicted
> r11  recent  false          0.18  contradicted                0.00  contradicted
> r12  recent  false          0.10  contradicted                0.00  contradicted
> r13  recent  false          0.03  contradicted                0.00  contradicted
> r14  recent  false          0.00  contradicted                0.00  contradicted
> r15  recent  false          0.03  contradicted                0.00  contradicted
> r16  recent  true           0.02  contradicted                1.00  supported           flip
>
> claim text in the search query: 24 of 24
> ungrounded correct 15/24  wrong 8  review 1 (4%, bound 25%)  missed failures 0 (bound 0)  false contradictions 8 (bound 5%)  Brier 0.326  flips 9
>            mean input tokens 99  mean latency 0.43 s  grounded attempts 0  cost per 1,000 claims $0.0020  acceptance not met
> grounded   correct 24/24  wrong 0  review 0 (0%, bound 25%)  missed failures 0 (bound 0)  false contradictions 0 (bound 5%)  Brier 0.000  flips 9
>            mean input tokens 1906  mean latency 0.58 s  grounded attempts 24  mean search results 5.0  cost per 1,000 claims $1.0381  acceptance met
> grounded attempts = POSTs sent with mode set, an upper bound on billed searches.
> saved out/eval-before0-after0-20261104T174430.json
> ```

Both modes got the eight stable claims right with probabilities at `1.00` or at `0.15` and under. The 16 recent claims split the two modes. Ungrounded, the 9 true recent claims printed between `0.01` and `0.24`: 8 were marked `contradicted` and one, `r08`, went to `review`. The 7 false recent claims also came back low, `0.00` to `0.18`, which counts as correct, but it is the same behavior: without a search, every claim about late 2026 got a low probability, true or false. That is 15 of 24 correct, but 8 of the 9 true recent claims were marked `contradicted`, which fails the false-contradiction bound of 5%: `acceptance not met`.

Grounded, the same 24 requests printed `1.00` for every true claim and `0.00` for every false one. 24 of 24, no reviews, no wrong verdicts, all three bounds met, and 9 flips, all of them true recent claims moving to `supported`. `r01` is the Navi Pillay claim from the top of the page: `0.01` to `1.00`. Mean input tokens went from 99 to 1,906, because the evidence is now part of the `state`; individual grounded requests ranged from 1,213 to 3,143 tokens, with 5 results for every claim. Mean latency went from 0.43 s to 0.58 s in this run; the earlier `check` run averaged 0.50 s grounded, so the latency figures describe this session more than the mode.

## What the search sees

Two runs with the same 24 claims and the same 600 characters of `FILLER`, which differ in where the claim sits. `--pad-before 600` puts the filler first and the claim on the line after it; `--pad-after 600` puts the claim first and the filler after it. Moving the claim changes two things at once: the search prefix, and the order of the `state` the model reads. `claim_in_query` says the claim is outside every query in the first run and inside every query in the second, and the script warns before the first run sends anything:

> [accordion] Run both paddings
>
> ```bash theme={null}
> python factcheck.py eval claims.jsonl --pad-before 600
> python factcheck.py eval claims.jsonl --pad-after 600
> ```

```text theme={null}
warning: 24 of 24 claims are not in their search query; the search sees the question and the first bytes of the state.
```

> [accordion] Observed output: claim after the filler
>
> ```text theme={null}
> id   kind    label  p_ungrounded  verdict_ungrounded    p_grounded  verdict_grounded
> s01  stable  true           0.99  supported                   0.92  supported
> s02  stable  true           0.75  review                      0.15  contradicted        flip
> s03  stable  true           1.00  supported                   1.00  supported
> s04  stable  true           0.99  supported                   0.99  supported
> s05  stable  false          0.07  contradicted                0.03  contradicted
> s06  stable  false          0.00  contradicted                0.00  contradicted
> s07  stable  false          0.00  contradicted                0.00  contradicted
> s08  stable  false          0.95  supported                   0.82  supported
> r01  recent  true           0.02  contradicted                0.07  contradicted
> r02  recent  true           0.26  review                      0.21  review
> r03  recent  true           0.33  review                      0.20  review
> r04  recent  true           0.21  review                      0.26  review
> r05  recent  true           0.07  contradicted                0.08  contradicted
> r06  recent  true           0.28  review                      0.07  contradicted        flip
> r07  recent  true           0.07  contradicted                0.12  contradicted
> r08  recent  true           0.39  review                      0.17  contradicted        flip
> r09  recent  false          0.10  contradicted                0.10  contradicted
> r10  recent  false          0.05  contradicted                0.07  contradicted
> r11  recent  false          0.10  contradicted                0.21  review              flip
> r12  recent  false          0.21  review                      0.11  contradicted        flip
> r13  recent  false          0.17  contradicted                0.05  contradicted
> r14  recent  false          0.01  contradicted                0.03  contradicted
> r15  recent  false          0.13  contradicted                0.16  contradicted
> r16  recent  true           0.12  contradicted                0.23  review              flip
>
> claim text in the search query: 0 of 24
> ungrounded correct 12/24  wrong 5  review 7 (29%, bound 25%)  missed failures 1 (bound 0)  false contradictions 4 (bound 5%)  Brier 0.294  flips 6
>            mean input tokens 212  mean latency 0.30 s  grounded attempts 0  cost per 1,000 claims $0.0042  acceptance not met
> grounded   correct 12/24  wrong 7  review 5 (21%, bound 25%)  missed failures 1 (bound 0)  false contradictions 6 (bound 5%)  Brier 0.330  flips 6
>            mean input tokens 915  mean latency 0.44 s  grounded attempts 24  mean search results 5.0  cost per 1,000 claims $1.0183  acceptance not met
> grounded attempts = POSTs sent with mode set, an upper bound on billed searches.
> saved out/eval-before600-after0-20261104T174507.json
> ```

> [accordion] Observed output: claim before the filler
>
> ```text theme={null}
> id   kind    label  p_ungrounded  verdict_ungrounded    p_grounded  verdict_grounded
> s01  stable  true           0.99  supported                   1.00  supported
> s02  stable  true           0.99  supported                   1.00  supported
> s03  stable  true           1.00  supported                   1.00  supported
> s04  stable  true           1.00  supported                   0.99  supported
> s05  stable  false          0.05  contradicted                0.00  contradicted
> s06  stable  false          0.00  contradicted                0.00  contradicted
> s07  stable  false          0.00  contradicted                0.00  contradicted
> s08  stable  false          0.23  review                      0.00  contradicted        flip
> r01  recent  true           0.01  contradicted                1.00  supported           flip
> r02  recent  true           0.03  contradicted                1.00  supported           flip
> r03  recent  true           0.03  contradicted                1.00  supported           flip
> r04  recent  true           0.06  contradicted                1.00  supported           flip
> r05  recent  true           0.03  contradicted                1.00  supported           flip
> r06  recent  true           0.06  contradicted                1.00  supported           flip
> r07  recent  true           0.04  contradicted                1.00  supported           flip
> r08  recent  true           0.24  review                      1.00  supported           flip
> r09  recent  false          0.01  contradicted                0.00  contradicted
> r10  recent  false          0.01  contradicted                0.00  contradicted
> r11  recent  false          0.09  contradicted                0.00  contradicted
> r12  recent  false          0.09  contradicted                0.00  contradicted
> r13  recent  false          0.01  contradicted                0.01  contradicted
> r14  recent  false          0.00  contradicted                0.00  contradicted
> r15  recent  false          0.03  contradicted                0.00  contradicted
> r16  recent  true           0.02  contradicted                0.99  supported           flip
>
> claim text in the search query: 24 of 24
> ungrounded correct 14/24  wrong 8  review 2 (8%, bound 25%)  missed failures 0 (bound 0)  false contradictions 8 (bound 5%)  Brier 0.338  flips 10
>            mean input tokens 212  mean latency 0.40 s  grounded attempts 0  cost per 1,000 claims $0.0042  acceptance not met
> grounded   correct 24/24  wrong 0  review 0 (0%, bound 25%)  missed failures 0 (bound 0)  false contradictions 0 (bound 5%)  Brier 0.000  flips 10
>            mean input tokens 1602  mean latency 0.52 s  grounded attempts 24  mean search results 5.0  cost per 1,000 claims $1.0320  acceptance met
> grounded attempts = POSTs sent with mode set, an upper bound on billed searches.
> saved out/eval-before0-after600-20261104T174530.json
> ```

With the claim after the filler, grounded accuracy fell from 24 of 24 to 12 of 24, with 5 claims in review and 5 of the 9 true recent claims marked `contradicted`; every `search_results` list in that run holds office-move pages. Even `s02`, "Rust 1.0 was released in May 2015", which both modes had at `1.00` unpadded, printed `0.15`. With the claim before the filler, the grounded run got 24 of 24 and met all three bounds, the same verdicts as the unpadded run. The experiment does not separate the two effects: the first run changed both the evidence and the order of the `state`, and the ungrounded results moved too, 12 of 24 with the claim after the filler and 14 of 24 with it before, against 15 of 24 unpadded. `s08`, "PostgreSQL is licensed under the GNU General Public License", went from `0.15` unpadded to `0.95` ungrounded with the filler in front of it, a missed failure, and `0.82` grounded. What the recorded outputs do show is that the claim's position decided what the search returned, and the run with the claim first is the one that met the bounds.

To see the wrong evidence with your own eyes, run `check` on one buried claim. The URLs under the row are the pages the search returned for a query that began with the office move:

> [accordion] Check one buried claim
>
> ```bash theme={null}
> grep '"r01"' claims.jsonl > r01.jsonl
> python factcheck.py check r01.jsonl --grounded --pad-before 600
> ```
>
> ```text theme={null}
> warning: 1 of 1 claims are not in their search query; the search sees the question and the first bytes of the state.
> id   kind    label    p_grounded  verdict_grounded
> r01  recent  true           0.07  contradicted
>                   https://kadence.co/templates/office-move-checklist/
>                   https://www.deskbird.com/blog/product-updates-q3-2026
>                   https://moving-company-guide.com/office-relocation/office-relocation-checklist
>                   https://nilsonvan.com/blog/office-move-checklist/
>                   https://www.goborntomove.com/blogs/office-moving-checklist-week-by-week-plan
>
> claim text in the search query: 0 of 1
> grounded   correct 0/1  wrong 1  review 0 (0%, bound 25%)  missed failures 0 (bound 0)  false contradictions 1 (bound 5%)  Brier 0.865
>            mean input tokens 913  mean latency 0.65 s  grounded attempts 1  mean search results 5.0  cost per 1,000 claims $1.0183  acceptance not met
> grounded attempts = POSTs sent with mode set, an upper bound on billed searches.
> saved out/check-before600-after0-20261104T174530.json
> ```

## Read the numbers

* **Recent against stable.** Grounding changed no verdict on the 8 stable claims. If your claims are about things settled years ago, measure before you pay for the search.
* **A low ungrounded probability is not a contradiction.** Ungrounded, the 9 true recent claims printed between `0.01` and `0.24` and the 7 false ones between `0.00` and `0.18`. Nothing in those numbers separates "false" from "no evidence either way". Treat `contradicted` from an ungrounded request as "unknown" for claims about recent events.
* **Evidence about the wrong thing looks like evidence.** With the claim buried, every grounded response carried office-move URLs and a verdict that looked as definite as any other; `search_results` is the only place in the response that shows what the evidence was. Keep the URLs in your records, as `out/` does here, and look at them when a verdict surprises you.
* **Grounding fails open, and the response says so.** This recipe sends one question, so one search per request: if it fails, runs out of time (1 second per search, 3 seconds for the request), or returns nothing, the model answers from your original `state` and `search_results` is `[]`. With several questions, the searches that succeed are used and `search_results` is `[]` only when none return. The script records the URLs per claim and prints `mean search results` on the grounded line, so a run where evidence went missing shows up in the summary; in the recorded runs it was `5.0` every time, 5 results for every claim.
* **Run-to-run drift.** We ran `eval` twice, back to back. The second run held all 48 verdicts. Ungrounded, four printed probabilities changed by `0.01` at two decimals (`r02` and `r04` from `0.04` to `0.03`, `r05` and `r12` from `0.10` to `0.11`), seven more changed by less than a hundredth, and 13 were identical to the last digit; grounded, none changed at two decimals and 12 were identical. Mean latency was 0.30 s and 0.49 s against 0.43 s and 0.58 s. The nearest call was `r11` ungrounded at `0.18`, two hundredths under the `contradicted` cutoff; a claim that settles that close to a cutoff can print a different verdict on another day, which the [ticket triage recipe](https://docs.perplexity.ai/docs/cookbook/examples/decisions-api-ticket-triage/README.md) saw happen.
* **Probabilities are the model's estimate.** 24 of 24 is one recorded run, not a guarantee. The verdict is only as good as the evidence the search found and the model's reading of it. Keep `review` in your policy and keep the labels on your side.

## Pricing

A grounded request bills input tokens at the model's rate plus \$1.00 per 1,000 searches, billed when the search returns, even with zero results and even if the decision then fails. In the recorded run, grounded requests averaged 1,906 input tokens: 1,906 × \$0.02 per million = \$0.000038 in tokens, plus \$0.001 for the search, or \$0.001038 per claim and \$1.0381 per 1,000 claims. Ungrounded requests averaged 99 tokens, or \$0.0020 per 1,000 claims. The search is 96% of the grounded cost: at the same input-token count, halving the token rate with the flash model (`--model pplx-decider-flash-latest`, \$0.01 per million) saves about two cents per 1,000. No flash run was recorded for this page, so measure it on your claims before relying on that figure or its accuracy.

A search that failed or timed out is not billed, and `search_results` does not say which searches ran, so the script prices every grounded attempt, retries included, and reports `grounded attempts` as an upper bound on billed searches. See [Pricing](https://docs.perplexity.ai/docs/getting-started/pricing.md#web-grounding).

## Next steps

* **Your claims.** Replace `claims.jsonl`. Put the claim first in the `state`, keep the question short, and run `check` on a sample: the warning and the `claim text in the search query` count tell you before any request whether each complete claim is in its query, and the URLs under each row tell you what the search found.
* **Several questions.** Each question gets its own query, up to 100 questions and 10 searches at a time; questions with the same query share a search, and empty queries are skipped. Each search returns up to 5 results and the request keeps up to 10, or one per query when there are more than 10 queries, ordered by rank. The question part of each query is capped at 128 bytes when the `state` has text. [Web grounding](https://docs.perplexity.ai/docs/decisions/quickstart.md#web-grounding) has the full rules, including criteria, `choice` and `score` questions, and object and array `state`.
* **Your cutoffs.** Change `SUPPORTED_MIN` and `CONTRADICTED_MAX`, rerun `eval`, and watch all three bounds in the summary. The JSON in `out/` has every probability and every URL, so you can replay cutoffs without new requests.
* **Read the docs.** [Web grounding](https://docs.perplexity.ai/docs/decisions/quickstart.md#web-grounding) covers the query, result, time, and failure rules in full; the [Decisions API quickstart](https://docs.perplexity.ai/docs/decisions/quickstart.md) covers the question types.