> ## Documentation Index
> Fetch the complete documentation index at: https://docs.perplexity.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Catch Visual Bugs in CI with the Decisions API

> Use Playwright screenshots and Decisions API probabilities to check selected pages for visual defects. Your code applies cutoffs, reports warnings, and fails CI when a failure rule fires.

A pull request can break your mobile pricing page while desktop tests pass; this recipe uses Decisions API probabilities from Playwright screenshots so your code can flag the defect in CI. Pixel diffs also highlight intentional font and color changes, which can make real bugs harder to spot. Here you ask specific questions about clipped text, overlap, and missing content, then apply explicit cutoffs rather than parse a free-text judgment.

The Decisions API returns a probability instead of a paragraph. Your CI step screenshots each page, sends the picture to `pplx-decider-v1.2` with six yes-or-no questions ("Is any text clipped?", "Do elements overlap?", "Is the call to action visible?"), and gets six numbers from 0 to 1. For a pull request it also sends the main and pull-request screenshots together and asks whether content went missing and whether the change looks deliberate. Your code compares the numbers to cutoffs, writes a Markdown report into the GitHub Actions job summary with the probabilities and the rule that fired, and exits nonzero when a failure rule fires. The model reads the screenshots and returns the probabilities; your code fails the build.

This recipe is three Python files, a small sample site with a landing page and a pricing page, and a GitHub Actions workflow. The sample site has three stylesheets: `main`, a pull request that breaks the mobile pricing table, and a deliberate redesign. The recorded runs below fail the broken variant and pass the redesign; they are fixture checks, not a guarantee for your site.

## Understand why a probability beats a pixel diff

* **Numbers you can threshold.** `text_clipped` came back at 96% for the broken mobile pricing page. Your code can compare that probability to an 80% cutoff without parsing a free-text judgment.
* **It reads meaning, not pixels.** A redesign that changed the hero to dark navy, recolored every button, and rounded the corners scored `content_lost` 0% and `intentional` 99%. A pixel diff of the same change is red on every page.
* **Six questions, one request.** `pplx-decider-v1.2` answers all six questions about one screenshot in one request and bills the screenshot once. The recorded run below includes the actual input-token totals.
* **Two images, two questions.** `state` is an array, so the before-and-after pair goes in one request with captions between the images. The model sees them in the order you write them.

## See what you will build

A command-line tool that serves the sample site, screenshots it, asks the questions, judges the answers, and writes the report. The same tool takes `--url` and `--baseline-url` for a site that is already running, such as a preview deployment against production.

<video autoPlay muted loop playsInline controls src="https://mintcdn.com/perplexity/LF01SxfG4XAyyxyj/docs/assets/images/cookbook/examples/decisions-api-visual-qa-gate-demo.mp4?fit=max&auto=format&n=LF01SxfG4XAyyxyj&q=85&s=79a46c2fe68cacf9acc3a3bd841fb46e" />

A recorded run, under a minute. A pull request drops the mobile media query, so the pricing page breaks at 390 px while desktop looks fine. A pixel diff flags the PR (11% changed) and a deliberate redesign (23% changed) alike. The decision model answers six yes/no questions per screenshot: the PR fails on text\_clipped 96% and content\_lost 96%, the redesign passes with intentional 94% and content\_lost 4%. Exit codes 1 and 0. Model: pplx-decider-v1.2.

```text theme={null}
decisions-api-visual-qa-gate/
├── requirements.txt       # the three packages, pinned
├── pyproject.toml         # settings for ruff, mypy, and pytest
├── site/
│   ├── index.html         # the landing page
│   ├── pricing.html       # the pricing page with three plan columns
│   ├── styles.css         # shared layout, served as /styles.css
│   ├── main.css           # what is deployed: the media query that stacks the columns on phones
│   ├── pr.css             # the pull request: that media query is gone
│   └── redesign.css       # a deliberate redesign: dark hero, new colors, nothing broken
├── decisions_client.py    # the eight questions, the request bodies, one request with retries
├── policy.py              # the cutoffs, the verdicts, and the Markdown report
├── visual_qa.py           # the local server, the screenshots, the loop, and the command line
├── test_visual_qa.py      # offline policy, response, screenshot, and report tests
├── visual-qa.yml          # the GitHub Actions workflow, to copy into .github/workflows/
└── out/                   # created by the script: screenshots, report.md, results.json
```

One run does the same thing for every page at every width:

1. `visual_qa.py` serves `site/` on a free local port, with the chosen variant's stylesheet as `/theme.css`.
2. Playwright loads the page at 1280 px and at 390 px and takes a full-page screenshot of each.
3. `decisions_client.py` sends the screenshot and six questions to the Decisions API and gets six probabilities back.
4. With a baseline, Playwright screenshots the baseline too, and `decisions_client.py` sends both images with two more questions.
5. `policy.py` compares every probability to the cutoffs, names the rule that fired, and renders the report.
6. `visual_qa.py` writes `out/report.md`, appends it to `$GITHUB_STEP_SUMMARY` when that variable is set, and exits 1 if any verdict is `fail`.

The model never sees the HTML, the CSS, or the cutoffs. It gets a picture and the questions and returns a probability for each.

## Gather what you need

* Python 3.10 or newer on macOS or Linux. The commands below use Unix shell syntax.
* A Perplexity API key from the [API Console](https://console.perplexity.ai/project/keys).
* About 700 MB of free disk space for the Chromium build that Playwright downloads.

## Set up the project

Create the folder, save `requirements.txt` and `pyproject.toml` in it, then run the remaining commands.

> **requirements.txt**
>
> ```text requirements.txt theme={null}
> httpx==0.28.1
> playwright==1.64.0
> pytest==9.1.1
> ```

`httpx` sends the API requests. `playwright` controls the browser. `pytest` runs the tests. `pyproject.toml` holds settings for `ruff` (code style), `mypy` (type checks), and `pytest`. You do not need `ruff` or `mypy` to run the gate.

> **pyproject.toml**
>
> ```toml pyproject.toml theme={null}
> [tool.ruff]
> line-length = 150
> target-version = "py310"
>
> [tool.ruff.lint]
> select = ["E", "F", "I", "B", "UP", "C4", "SIM"]
>
> [tool.mypy]
> strict = true
> ignore_missing_imports = true
>
> [tool.pytest.ini_options]
> addopts = "-q"
> ```

> **Install and set your key**
>
> ```bash theme={null}
> python3 --version   # 3.10 or newer
> mkdir decisions-api-visual-qa-gate && cd decisions-api-visual-qa-gate
> # save requirements.txt and pyproject.toml in this folder, then continue
> python3 -m venv .venv
> source .venv/bin/activate
> python -m pip install -r requirements.txt
> python -m playwright install chromium
> export PERPLEXITY_API_KEY="your-api-key-here"
> ```

The Chromium download is the slowest step. In a new terminal, run the `source` and `export` lines again.

## Explore the sample site

Six files in `site/`. Both pages link two stylesheets: `/styles.css`, which is the shared layout, and `/theme.css`, which the local server maps to `main.css`, `pr.css`, or `redesign.css` depending on the variant you ask for. That is the whole mechanism for "main versus this pull request": same HTML, different stylesheet.

> **site/index.html**
>
> ```html site/index.html theme={null}
> <!doctype html>
> <html lang="en">
> <head>
>   <meta charset="utf-8">
>   <meta name="viewport" content="width=device-width, initial-scale=1">
>   <title>Ledgerline: invoicing for small teams</title>
>   <link rel="stylesheet" href="/styles.css">
>   <link rel="stylesheet" href="/theme.css">
> </head>
> <body>
>   <header class="nav">
>     <a class="brand" href="/">Ledgerline</a>
>     <nav><a href="/pricing.html">Pricing</a><a href="#features">Features</a><a href="#" class="button small">Sign in</a></nav>
>   </header>
>   <main>
>     <section class="hero">
>       <h1>Invoices out in minutes, paid in days</h1>
>       <p>Ledgerline sends, tracks, and reconciles invoices for teams of two to fifty. No spreadsheets, no chasing.</p>
>       <a href="/pricing.html" class="button primary">Start free trial</a>
>     </section>
>     <section id="features" class="features">
>       <article><h2>Send</h2><p>Branded invoices with your terms, due dates, and payment links.</p></article>
>       <article><h2>Track</h2><p>See who opened what, and get a nudge before an invoice goes late.</p></article>
>       <article><h2>Reconcile</h2><p>Match payments to invoices automatically and export to your books.</p></article>
>     </section>
>   </main>
>   <footer>Ledgerline Inc. All amounts shown in USD.</footer>
> </body>
> </html>
> ```

> **site/pricing.html**
>
> ```html site/pricing.html theme={null}
> <!doctype html>
> <html lang="en">
> <head>
>   <meta charset="utf-8">
>   <meta name="viewport" content="width=device-width, initial-scale=1">
>   <title>Ledgerline pricing</title>
>   <link rel="stylesheet" href="/styles.css">
>   <link rel="stylesheet" href="/theme.css">
> </head>
> <body>
>   <header class="nav">
>     <a class="brand" href="/">Ledgerline</a>
>     <nav><a href="/pricing.html">Pricing</a><a href="/#features">Features</a><a href="#" class="button small">Sign in</a></nav>
>   </header>
>   <main>
>     <section class="hero compact">
>       <h1>Simple pricing, billed monthly</h1>
>       <p>Every plan includes unlimited invoices and payment links. Cancel any time.</p>
>     </section>
>     <section class="plans">
>       <article class="plan">
>         <h2>Starter</h2>
>         <p class="price">$12<span>/month</span></p>
>         <ul><li>1 user</li><li>Unlimited invoices</li><li>Email reminders</li></ul>
>         <a href="#" class="button">Choose Starter</a>
>       </article>
>       <article class="plan featured">
>         <h2>Team</h2>
>         <p class="price">$39<span>/month</span></p>
>         <ul><li>Up to 10 users</li><li>Automatic reconciliation</li><li>Accounting export</li></ul>
>         <a href="#" class="button primary">Choose Team</a>
>       </article>
>       <article class="plan">
>         <h2>Business</h2>
>         <p class="price">$99<span>/month</span></p>
>         <ul><li>Up to 50 users</li><li>Approval workflows</li><li>Priority support</li></ul>
>         <a href="#" class="button">Choose Business</a>
>       </article>
>     </section>
>   </main>
>   <footer>Ledgerline Inc. All amounts shown in USD.</footer>
> </body>
> </html>
> ```

> **site/styles.css**
>
> ```css site/styles.css theme={null}
> /* Shared layout. The variant file served as /theme.css adds the responsive rules and the look. */
> * { box-sizing: border-box; }
> body { margin: 0; font: 16px/1.5 -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif; color: #1f2933; background: #fff; }
> .nav { display: flex; justify-content: space-between; align-items: center; padding: 16px 24px; border-bottom: 1px solid #e4e7eb; }
> .nav nav { display: flex; gap: 20px; align-items: center; }
> .nav a { color: #1f2933; text-decoration: none; }
> .brand { font-weight: 700; font-size: 20px; }
> main { max-width: 1040px; margin: 0 auto; padding: 0 24px; }
> .hero { text-align: center; padding: 72px 0 48px; }
> .hero.compact { padding: 48px 0 24px; }
> .hero h1 { font-size: 44px; line-height: 1.1; margin: 0 0 16px; }
> .hero p { font-size: 18px; color: #52606d; max-width: 560px; margin: 0 auto 28px; }
> .button { display: inline-block; padding: 12px 22px; border-radius: 8px; border: 1px solid #1f2933; text-decoration: none; color: #1f2933; font-weight: 600; }
> .button.primary { background: #0b57d0; border-color: #0b57d0; color: #fff; }
> .button.small { padding: 6px 14px; }
> .features { display: grid; grid-template-columns: repeat(auto-fit, minmax(240px, 1fr)); gap: 24px; padding: 24px 0 64px; }
> .features article { padding: 24px; border: 1px solid #e4e7eb; border-radius: 12px; }
> .features h2 { margin: 0 0 8px; font-size: 22px; }
> .plans { display: grid; grid-template-columns: repeat(3, minmax(0, 1fr)); gap: 24px; padding: 24px 0 64px; }
> .plan { min-width: 280px; padding: 28px; border: 1px solid #e4e7eb; border-radius: 12px; background: #fff; }
> .plan.featured { border-color: #0b57d0; box-shadow: 0 8px 24px rgba(11, 87, 208, 0.12); }
> .plan h2 { margin: 0 0 8px; font-size: 24px; }
> .price { font-size: 40px; font-weight: 700; margin: 0 0 16px; }
> .price span { font-size: 16px; font-weight: 400; color: #52606d; }
> .plan ul { padding-left: 20px; margin: 0 0 24px; color: #52606d; }
> footer { text-align: center; color: #7b8794; padding: 24px; border-top: 1px solid #e4e7eb; font-size: 14px; }
> ```

The defect is in two lines of `styles.css`. `.plans` is a three-column grid with `minmax(0, 1fr)` tracks, and each `.plan` has `min-width: 280px`. On a 1280 px screen the tracks are about 315 px wide and nothing happens. At 390 px each track is about 100 px, every card insists on 280 px, and the grid lets them overlap. `main.css` fixes that with one media query. `pr.css` is what a cleanup commit left behind after removing it. `redesign.css` keeps the media query and changes the look.

> **site/main.css**
>
> ```css site/main.css theme={null}
> /* What is deployed today. The media query stacks the plan columns on narrow screens. */
> @media (max-width: 720px) {
>   .hero h1 { font-size: 32px; }
>   .plans { grid-template-columns: 1fr; }
>   .plan { min-width: 0; }
> }
> ```

> **site/pr.css**
>
> ```css site/pr.css theme={null}
> /* The pull request under test. A cleanup removed the "unused" media query from main.css, so the
>    three plan columns keep their 280 px minimum width at 390 px and draw on top of each other. */
> .hero h1 { letter-spacing: -0.01em; }
> ```

> **site/redesign.css**
>
> ```css site/redesign.css theme={null}
> /* A deliberate redesign: dark hero, new headline weight, rounder buttons. Nothing is broken. */
> @media (max-width: 720px) {
>   .hero h1 { font-size: 32px; }
>   .plans { grid-template-columns: 1fr; }
>   .plan { min-width: 0; }
> }
> .hero { background: #0f172a; color: #fff; border-radius: 16px; padding: 64px 24px 56px; margin-top: 24px; }
> .hero.compact { padding: 48px 24px 32px; }
> .hero h1 { font-weight: 800; }
> .hero p { color: #cbd5e1; }
> .button { border-radius: 999px; }
> .button.primary { background: #22c55e; border-color: #22c55e; color: #052e16; }
> .plan.featured { border-color: #22c55e; box-shadow: 0 8px 24px rgba(34, 197, 94, 0.18); }
> ```

Save all six in `site/` inside the project folder.

## Choose the cutoffs before writing code

The gate maps probabilities to three outcomes: `fail` stops the build, `warn` passes the build but asks a person to look at the screenshot, and `pass` is silent. Before writing the gate, decide what those cutoffs should be and what a good result looks like.

Two things can go wrong. The gate can miss a real defect, which is the bug the recipe exists to catch. Or it can flag a clean screenshot, which is the reason teams stop reading pixel-diff reports. The acceptance criteria bound both:

* At most 10% of defects pass silently.
* At most 1% of clean screenshots fail the build.
* At most 10% of clean screenshots are flagged at all (`warn` or `fail`).

`test_visual_qa.py` checks these on a synthetic distribution of 20,000 screenshots: 85% clean, with defect questions mostly near zero and a noisy tail that crosses 40% on a few percent of them; 15% carrying one defect, three quarters of them obvious (probabilities around 85%) and one quarter subtle (spread around 50%). The baseline to beat is the trivial policy, review everything: it misses nothing and flags everything, so if the gate cannot do far better it is not worth running.

| Policy | Defects missed | Clean screenshots failed | Clean screenshots flagged | Screenshots a person looks at |
| - | - | - | - | - |
| Review everything | 0% | 0% | 100% | 100% |
| Fail at 50% | 12.2% | 3.1% | 3.1% | 15.6% |
| Fail at 80% | 43.2% | 0.0% | 0.0% | 8.4% |
| Fail at 90% | 71.0% | 0.0% | 0.0% | 4.3% |
| Fail at 80%, warn at 40% | 7.3% | 0.0% | 7.7% | 20.3% |

One cutoff forces a trade: low enough to catch the subtle defects and it fails clean builds, high enough to never fail a clean build and it misses almost half the defects. Two cutoffs take both ends. The build fails only on a probability of 80% or more, and anything from 40% up shows in the report as a warning with the screenshot path. That is the policy in `policy.py`: `FAIL_AT = 0.80`, `WARN_AT = 0.40`. `cta_visible` is the one question where a high probability is good, so its cutoffs are mirrored: fail at or below 20%, warn at or below 60%.

The before-and-after pair has its own rule. `content_lost` at or above 80% with `intentional` at or below 40% is a regression and fails. `content_lost` at or above 40% with anything else is a warning: a deliberate removal still deserves a look, but it should not stop the build. The second synthetic test runs 20,000 pairs through that rule with 10% regressions, 15% redesigns (some of which remove content on purpose), and 75% unchanged pages, and holds the same three bounds.

The distribution is synthetic. It stands in for the labeled screenshots you do not have yet; once you do, replace the generator in the tests with your own probabilities and keep the acceptance criteria.

## Write the API client: decisions\_client.py

`decisions_client.py` calls the Decisions API and knows nothing about browsers or cutoffs. Copy the complete file from [Copy the complete files](#copy-the-complete-files), then read it here in four parts.

### Configure the endpoint, model, and retry settings

> **Endpoint, model, and retry settings**
>
> ```python theme={null}
> """The part of the gate that talks to the Decisions API: the questions, the request bodies, and
> bounded retries. Nothing in this file knows about browsers, pages, or cutoffs."""
>
> from __future__ import annotations
>
> import base64
> import time
> from typing import Any
>
> import httpx
>
> API_URL = "https://api.perplexity.ai/v1/decisions"
> MODEL = "pplx-decider-v1.2"  # groups the questions and bills the screenshot once
> REQUEST_TIMEOUT_S = 90.0  # a 1,000-tile screenshot answers in about a second; this covers a slow one
> RETRY_STATUSES = {429, 500, 502, 503, 504}
> MAX_RETRIES = 2  # two more tries for those and for dropped connections, waiting Retry-After seconds (or 1 s, then 2 s)
> MAX_RETRY_DELAY_S = 30.0  # if Retry-After asks for more than this, stop with an error instead of waiting
> ```

* `MODEL` is `pplx-decider-v1.2`. Flash processes grouped questions separately and bills the state for each question, so a lower input-token price does not mean a lower cost for this six-question request. Keep this recipe on `pplx-decider-v1.2`; see [Grouping questions in one request](https://docs.perplexity.ai/docs/decisions/quickstart.md#grouping-questions-in-one-request).
* `REQUEST_TIMEOUT_S`, `RETRY_STATUSES`, `MAX_RETRIES`, and `MAX_RETRY_DELAY_S` control the timeout and retries. A 429 (rate limit), a 5xx (server error), or a dropped connection is retried twice. A 400 or 401 is a problem with the request and is not retried. If the server asks for a wait longer than 30 seconds, the script stops with an error rather than retry early.

### Define the eight questions

> **The eight questions**
>
> ```python theme={null}
> # Six yes-or-no questions about one screenshot. The answer to each is the probability of yes.
> SINGLE_QUESTIONS: dict[str, dict[str, Any]] = {
>     "text_clipped": {"type": "noul", "instructions": "Is any text clipped, cut off mid-word, or drawn on top of other text?"},
>     "elements_overlap": {
>         "type": "noul",
>         "instructions": "Do any elements such as cards, images, or buttons overlap or cover other content that should be visible?",
>     },
>     "cta_visible": {
>         "type": "noul",
>         "instructions": "Is a primary call-to-action button, such as Start free trial, Choose a plan, Get started, or Buy, "
>         "fully visible and readable?",
>     },
>     "low_contrast": {"type": "noul", "instructions": "Is any text hard to read because it has too little contrast with its background?"},
>     "layout_broken": {
>         "type": "noul",
>         "instructions": "Is the layout broken, with elements pushed off-screen, misaligned, stacked wrongly, or a horizontal scrollbar?",
>     },
>     "error_state": {"type": "noul", "instructions": "Does the page look like a loading, blank, or error state rather than a finished page?"},
> }
>
> # Two questions about a before-and-after pair. BEFORE is the baseline, AFTER is the change under review.
> PAIR_QUESTIONS: dict[str, dict[str, Any]] = {
>     "content_lost": {
>         "type": "noul",
>         "instructions": "Is content that is present in the BEFORE screenshot (text, buttons, or whole sections) missing or hidden "
>         "in the AFTER screenshot?",
>     },
>     "intentional": {
>         "type": "noul",
>         "instructions": "Do the differences between BEFORE and AFTER look like a deliberate design change rather than a rendering bug?",
>     },
> }
> ```

All eight are `noul` questions: the answer to each is the probability of yes. The six in `SINGLE_QUESTIONS` go with one screenshot. Five of them describe defects, so a high probability is bad. `cta_visible` is the exception: it asks whether the main button is fully visible, so a low probability is bad. The two in `PAIR_QUESTIONS` go with two screenshots and refer to them as BEFORE and AFTER, the same words the request puts in front of each image.

The questions name no product, page, or site. They describe what a broken page looks like in general, so the same six questions work on the sample site and on yours.

### Build the request bodies

> **Request bodies**
>
> ```python theme={null}
> class DecisionsError(Exception):
>     """The Decisions API returned an error response, or a response the script cannot read."""
>
>
> def make_client(api_key: str) -> httpx.Client:
>     """One HTTP client for the whole run: it carries the key, the timeout, and a reused connection."""
>     return httpx.Client(headers={"Authorization": f"Bearer {api_key}"}, timeout=REQUEST_TIMEOUT_S)
>
>
> def image_part(png: bytes) -> dict[str, Any]:
>     """One screenshot as the OpenAI-style image part the Decisions API reads inside state."""
>     return {"type": "image_url", "image_url": {"url": "data:image/png;base64," + base64.b64encode(png).decode()}}
>
>
> def single_body(png: bytes, label: str) -> dict[str, Any]:
>     """The request for one screenshot: a caption, the image, and the six questions."""
>     return {"model": MODEL, "state": [f"Screenshot of {label}.", image_part(png)], "questions": SINGLE_QUESTIONS}
>
>
> def pair_body(before: bytes, after: bytes, label: str) -> dict[str, Any]:
>     """The request for a before-and-after pair. Order matters: the model reads state top to bottom,
>     so each caption comes right before its image."""
>     return {
>         "model": MODEL,
>         "state": [f"Two screenshots of {label}.", "BEFORE (baseline):", image_part(before), "AFTER (change under review):", image_part(after)],
>         "questions": PAIR_QUESTIONS,
>     }
> ```

`image_part` wraps a PNG as the OpenAI-style image part the Decisions API reads: `{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}`. PNG, JPEG, and WebP data URLs are accepted; `http` URLs return 400.

`single_body` builds the `state` as a two-item array: a one-line caption such as "Screenshot of pricing at 390 px." and the image. `pair_body` builds a five-item array: a caption, the word BEFORE, the baseline image, the word AFTER, the new image. The model reads `state` top to bottom with each image placed where its part appears, so the captions tell it which picture is which.

### Send one request with retries

> **One request, with retries**
>
> ```python theme={null}
> def ask(client: httpx.Client, body: dict[str, Any]) -> tuple[dict[str, float], int, str | None]:
>     """Send one request and return the probability of yes per question, the input tokens, and the request id.
>     Retries 429, 5xx, and dropped connections a bounded number of times; raises DecisionsError on any other failure."""
>     for attempt in range(MAX_RETRIES + 1):
>         try:
>             response = client.post(API_URL, json=body)
>         except httpx.TransportError:  # dropped connection or read timeout
>             if attempt == MAX_RETRIES:
>                 raise
>             time.sleep(float(2**attempt))
>             continue
>         rid = response.headers.get("x-request-id")
>         if response.status_code == 200:
>             return _parse(response, body["questions"], rid)
>         if response.status_code not in RETRY_STATUSES or attempt == MAX_RETRIES:
>             break
>         delay = _retry_delay(response, attempt)
>         if delay > MAX_RETRY_DELAY_S:
>             raise DecisionsError(f"Decisions API asked to wait {delay:.0f} s before retrying ({response.status_code}); stopping instead")
>         time.sleep(delay)
>     raise DecisionsError(f"Decisions API failed: {response.status_code} {response.text.strip()}" + (f" (x-request-id {rid})" if rid else ""))
>
>
> def _parse(response: httpx.Response, questions: dict[str, Any], rid: str | None) -> tuple[dict[str, float], int, str | None]:
>     """The probabilities out of a 200, checked for every question this script asked. Anything else is a DecisionsError."""
>     try:
>         data = response.json()
>         probabilities = {name: float(data["answers"][name]["noul"]) for name in questions}
>         if any(not 0 <= p <= 1 for p in probabilities.values()):
>             raise ValueError("probabilities must be finite numbers from 0 to 1")
>         return probabilities, int(data["usage"]["input_tokens"]), rid
>     except (ValueError, KeyError, TypeError) as exc:
>         raise DecisionsError(
>             f"Decisions API returned a 200 this script cannot read ({exc!r}): {response.text[:200]!r}" + (f" (x-request-id {rid})" if rid else "")
>         ) from exc
>
>
> def _retry_delay(response: httpx.Response, attempt: int) -> float:
>     """Seconds to wait before retrying: the Retry-After header if it is a number, else 1 s then 2 s."""
>     try:
>         return max(0.0, float(response.headers.get("retry-after", "")))
>     except ValueError:
>         return float(2**attempt)
> ```

`ask` sends one request and returns three things: a dictionary of probabilities keyed by question name, the input tokens from `usage`, and the `x-request-id` header. It takes the `httpx.Client` as an argument rather than creating its own, so the tests can pass a fake client that returns prepared responses.

The loop makes up to three attempts. A 200 returns the parsed answers. A 429, 5xx, or dropped connection waits and retries; `_retry_delay` uses the `Retry-After` header when the server sends a number, otherwise 1 second, then 2. Any other status raises `DecisionsError` with the status code, the API's message, and the request id. Include that id in any support request.

`_parse` reads `answers[name]["noul"]` for every question the request asked and `usage.input_tokens`. A 200 missing any of them becomes a `DecisionsError` instead of a `KeyError` later.

## Write the policy and report: policy.py

`policy.py` turns probabilities into verdicts and verdicts into Markdown. It imports nothing from the other two files. Copy the complete file from [Copy the complete files](#copy-the-complete-files), then read it here in three parts.

### Turn cutoffs into a verdict

> **Cutoffs and the verdict**
>
> ```python theme={null}
> """The gate's policy and report. The model returns probabilities; this file compares them to cutoffs,
> names the rule that fired, and renders the Markdown that lands in the GitHub Actions job summary."""
>
> from __future__ import annotations
>
> from dataclasses import dataclass, field
>
> FAIL_AT = 0.80  # a defect question at or above this fails the build
> WARN_AT = 0.40  # at or above this (and below FAIL_AT) the build passes with a warning to look at the screenshot
> DEFECT_QUESTIONS = ["text_clipped", "elements_overlap", "low_contrast", "layout_broken", "error_state"]
> # cta_visible is the one question where a high probability is good, so its cutoffs are mirrored.
> CTA_FAIL_BELOW = round(1 - FAIL_AT, 2)
> CTA_WARN_BELOW = round(1 - WARN_AT, 2)
>
> ORDER = {"fail": 0, "warn": 1, "pass": 2}
>
>
> @dataclass
> class Verdict:
>     """One gate result: the screenshot it is about, the outcome, the rule that fired, and every probability."""
>
>     label: str  # "pricing.html at 390 px" or "pricing.html at 390 px (before/after)"
>     outcome: str  # pass, warn, or fail
>     rule: str  # the rule that fired, or "all clear"
>     probabilities: dict[str, float] = field(default_factory=dict)
>     screenshot: str = ""  # path of the AFTER screenshot, for the report
> ```

`FAIL_AT` and `WARN_AT` are the two cutoffs from [Choose the cutoffs before writing code](#choose-the-cutoffs-before-writing-code). `CTA_FAIL_BELOW` and `CTA_WARN_BELOW` are their mirror images for `cta_visible`. `ORDER` ranks the outcomes so the worst one can be found with `min`.

A `Verdict` is one row of the report: the screenshot it is about, the outcome, the rule that fired in plain words with the probability in it, every probability from the response, and the path of the screenshot so a reader of the report can open the picture.

### Judge one screenshot and one pair

> **Judging one screenshot and one pair**
>
> ```python theme={null}
> def judge_single(label: str, p: dict[str, float], screenshot: str = "") -> Verdict:
>     """Compare the six probabilities for one screenshot to the cutoffs. The worst defect question wins."""
>     worst = max(DEFECT_QUESTIONS, key=lambda q: p[q])
>     if p[worst] >= FAIL_AT:
>         return Verdict(label, "fail", f"{worst} {p[worst]:.0%} is at or above {FAIL_AT:.0%}", p, screenshot)
>     if p["cta_visible"] <= CTA_FAIL_BELOW:
>         return Verdict(label, "fail", f"cta_visible {p['cta_visible']:.0%} is at or below {CTA_FAIL_BELOW:.0%}", p, screenshot)
>     if p[worst] >= WARN_AT:
>         return Verdict(label, "warn", f"{worst} {p[worst]:.0%} is at or above {WARN_AT:.0%}; look at the screenshot", p, screenshot)
>     if p["cta_visible"] <= CTA_WARN_BELOW:
>         rule = f"cta_visible {p['cta_visible']:.0%} is at or below {CTA_WARN_BELOW:.0%}; look at the screenshot"
>         return Verdict(label, "warn", rule, p, screenshot)
>     return Verdict(label, "pass", "all clear", p, screenshot)
>
>
> def judge_pair(label: str, p: dict[str, float], screenshot: str = "") -> Verdict:
>     """Compare the two before-and-after probabilities to the cutoffs. Lost content fails only when the
>     change does not look deliberate; a deliberate removal is flagged for a person to confirm."""
>     lost, intentional = p["content_lost"], p["intentional"]
>     if lost >= FAIL_AT and intentional <= WARN_AT:
>         return Verdict(label, "fail", f"content_lost {lost:.0%} with intentional {intentional:.0%}: looks like a regression", p, screenshot)
>     if lost >= WARN_AT:
>         return Verdict(label, "warn", f"content_lost {lost:.0%} with intentional {intentional:.0%}; confirm the removal was meant", p, screenshot)
>     return Verdict(label, "pass", "all clear", p, screenshot)
>
>
> def overall(verdicts: list[Verdict]) -> str:
>     """The run's outcome is the worst verdict in it."""
>     return min((v.outcome for v in verdicts), key=lambda outcome: ORDER[outcome], default="pass")
> ```

`judge_single` finds the defect question with the highest probability and compares it to the cutoffs, then does the same for `cta_visible` on the mirrored scale. The rule string carries the question name and the probability, so the report says `text_clipped 96% is at or above 80%` rather than just "fail".

`judge_pair` checks `content_lost` first. Lost content with a low `intentional` probability is a regression and fails. Lost content with a high `intentional` probability is a warning for a person to confirm. If nothing was lost, `intentional` is not consulted at all; a recolored button is not a bug.

`overall` is the worst verdict in the run, and an empty run passes.

### Render the report

> **The report**
>
> ```python theme={null}
> def render_report(verdicts: list[Verdict], head: str, base: str | None, tokens: int, model: str) -> str:
>     """GitHub-flavored Markdown: a headline, one table row per verdict, and the probabilities behind each row."""
>     result = overall(verdicts)
>     lines = [f"## Visual QA: {result.upper()}", ""]
>     against = f" against baseline `{base}`" if base else ""
>     lines.append(f"Checked `{head}`{against} with `{model}`: {len(verdicts)} verdicts, {tokens:,} input tokens.")
>     lines += ["", "| Screenshot | Result | Rule |", "|---|---|---|"]
>     for v in verdicts:
>         lines.append(f"| {v.label} | {v.outcome} | {v.rule} |")
>     lines += ["", "<details><summary>Probabilities</summary>", ""]
>     for v in verdicts:
>         probs = ", ".join(f"{k} {p:.0%}" for k, p in v.probabilities.items())
>         lines.append(f"- **{v.label}**: {probs}" + (f" ({v.screenshot})" if v.screenshot else ""))
>     lines += ["", "</details>", ""]
>     return "\n".join(lines)
> ```

`render_report` writes GitHub-flavored Markdown: a headline with the overall result, one line with what was checked and the total input tokens, a table with one row per verdict, and a collapsed `<details>` block with every probability and screenshot path. GitHub renders all of this in the job summary, and the table is short enough to read on a phone.

## Write the gate: visual\_qa.py

`visual_qa.py` serves the site, takes the screenshots, runs the loop, and owns the command line. Copy the complete file from [Copy the complete files](#copy-the-complete-files), then read it here in five parts.

### Declare imports and constants

> **Imports and constants**
>
> ```python theme={null}
> """A visual QA gate for CI. Screenshots each page at two widths, sends every screenshot to the Decisions API
> with six yes-or-no questions, and (with a baseline) sends before-and-after pairs with two more. The model
> returns probabilities; policy.py compares them to cutoffs. The exit code is 1 when any verdict is fail.
>
>     python visual_qa.py --variant pr --baseline main        # the bundled site: a pull request against main
>     python visual_qa.py --url https://preview.example.com --baseline-url https://www.example.com
> """
>
> from __future__ import annotations
>
> import argparse
> import functools
> import json
> import os
> import sys
> import threading
> from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
> from pathlib import Path
> from typing import Any
> from urllib.parse import urlparse
>
> import httpx
> from playwright.sync_api import Browser, FloatRect, sync_playwright
> from playwright.sync_api import Error as PlaywrightError
>
> import decisions_client as dc
> import policy
>
> SITE_DIR = Path(__file__).parent / "site"
> VARIANTS = ["main", "pr", "redesign"]  # each one is site/<variant>.css, served as /theme.css
> VIEWPORTS = {"desktop": (1280, 800), "mobile": (390, 844)}
> MAX_TILES = 2048  # the Decisions API reads images in 32 x 32 tiles; a larger image waits about a minute and returns 504
> DEFAULT_PAGES = ["/", "/pricing.html"]
> ```

`VIEWPORTS` is desktop at 1280 by 800 and mobile at 390 by 844, the logical size of a recent iPhone. `MAX_TILES` is the Decisions API's limit of 2,048 tiles of 32 by 32 pixels per image. An image over that limit does not return 400; the request waits about a minute and returns 504, so the script keeps every screenshot under it.

### Serve the site locally

> **The local server**
>
> ```python theme={null}
> class SiteHandler(SimpleHTTPRequestHandler):
>     """Serves site/ as static files, except /theme.css, which comes from site/<variant>.css."""
>
>     variant = "main"
>
>     def translate_path(self, path: str) -> str:
>         if path.split("?")[0] == "/theme.css":
>             return str(SITE_DIR / f"{self.variant}.css")
>         return super().translate_path(path)
>
>     def log_message(self, format: str, *args: Any) -> None:  # noqa: A002 (the base class names it format)
>         pass  # keep the console for the gate's own output
>
>
> def serve(variant: str) -> tuple[ThreadingHTTPServer, str]:
>     """Start one local server for a variant on a free port. Returns the server and its base URL."""
>     handler = type(f"Handler_{variant}", (SiteHandler,), {"variant": variant})
>     server = ThreadingHTTPServer(("127.0.0.1", 0), functools.partial(handler, directory=str(SITE_DIR)))
>     threading.Thread(target=server.serve_forever, daemon=True).start()
>     return server, f"http://127.0.0.1:{server.server_address[1]}"
> ```

`SiteHandler` is Python's built-in static file handler with one change: a request for `/theme.css` is answered from `site/<variant>.css`. `serve` makes a subclass per variant, binds it to port 0 so the operating system picks a free port, and runs it on a daemon thread. With a baseline, two servers run at once, one per variant, and the gate shuts both down at the end. Logging is silenced so the console shows only the gate's own lines.

### Capture screenshots within the tile budget

> **Screenshots within the tile budget**
>
> ```python theme={null}
> def max_height(width: int) -> int:
>     """The tallest screenshot of this width that stays within MAX_TILES, as a multiple of 32 pixels."""
>     return (MAX_TILES // max(1, round(width / 32))) * 32
>
>
> def screenshot(browser: Browser, url: str, width: int, height: int) -> bytes:
>     """A full-page PNG. Reject pages over the tile budget rather than silently omit content."""
>     context = browser.new_context(viewport={"width": width, "height": height}, reduced_motion="reduce")
>     try:
>         page = context.new_page()
>         page.goto(url, wait_until="networkidle")
>         page.wait_for_timeout(250)  # web fonts and late layout
>         full = int(page.evaluate("document.documentElement.scrollHeight"))
>         if max(full, height) > max_height(width):
>             raise ValueError(f"{url}: page exceeds the image budget at {width} px; split it into sections before checking")
>         clip: FloatRect = {"x": 0, "y": 0, "width": width, "height": max(full, height)}
>         return page.screenshot(full_page=True, clip=clip, animations="disabled")
>     finally:
>         context.close()
>
>
> def slug(path: str) -> str:
>     """'/pricing.html' -> 'pricing', '/' -> 'index'."""
>     return path.strip("/").removesuffix(".html").replace("/", "-") or "index"
> ```

`max_height` turns the tile budget into a pixel height for a given width: 1,632 px at 1280 wide, 5,440 px at 390 wide. `screenshot` opens a fresh browser context with reduced motion and waits for network idle plus a quarter second. It captures the full height at the viewport width, so horizontal overflow does not widen the image beyond what a phone user sees. It stops with an error if the height exceeds the budget, rather than silently skip lower content. For longer pages, add section-by-section capture. Disabling CSS animations reduces variation; dynamic content and late-loading fonts can still change screenshots.

### Run the loop

> **The loop**
>
> ```python theme={null}
> def run(args: argparse.Namespace, api_key: str) -> int:
>     """Screenshot, ask, judge, report. Returns the process exit code."""
>     out = Path(args.out)
>     out.mkdir(exist_ok=True)
>     servers: list[ThreadingHTTPServer] = []
>     head_url, base_url = args.url, args.baseline_url
>     head_name = args.variant or urlparse(args.url).hostname or "head"  # names the screenshots and the report
>     base_name = args.baseline or (urlparse(args.baseline_url).hostname if args.baseline_url else None)
>     if args.variant:
>         server, head_url = serve(args.variant)
>         servers.append(server)
>     if args.baseline:
>         server, base_url = serve(args.baseline)
>         servers.append(server)
>
>     verdicts: list[policy.Verdict] = []
>     tokens = 0
>     try:
>         with sync_playwright() as pw, pw.chromium.launch() as browser, dc.make_client(api_key) as client:
>             for path in args.pages:
>                 for view, (width, height) in VIEWPORTS.items():
>                     label = f"{slug(path)} at {width} px"
>                     after = screenshot(browser, head_url + path, width, height)
>                     shot = out / f"{head_name}-{slug(path)}-{view}.png"
>                     shot.write_bytes(after)
>                     probs, used, rid = dc.ask(client, dc.single_body(after, label))
>                     tokens += used
>                     verdicts.append(policy.judge_single(label, probs, str(shot)))
>                     print(f"{label}: {verdicts[-1].outcome} ({verdicts[-1].rule}) [{used} tokens, {rid}]")
>                     if base_url:
>                         before = screenshot(browser, base_url + path, width, height)
>                         (out / f"{base_name}-{slug(path)}-{view}.png").write_bytes(before)
>                         probs, used, rid = dc.ask(client, dc.pair_body(before, after, label))
>                         tokens += used
>                         verdicts.append(policy.judge_pair(f"{label} (before/after)", probs, str(shot)))
>                         print(f"{label} (before/after): {verdicts[-1].outcome} ({verdicts[-1].rule}) [{used} tokens, {rid}]")
>     except (dc.DecisionsError, httpx.HTTPError, PlaywrightError, ValueError) as exc:
>         print(f"error: {exc}", file=sys.stderr)
>         return 2
>     finally:
>         for server in servers:
>             server.shutdown()
>
>     report = policy.render_report(verdicts, head_name, base_name, tokens, dc.MODEL)
>     (out / "report.md").write_text(report)
>     (out / "results.json").write_text(json.dumps([vars(v) for v in verdicts], indent=2))
>     if summary := os.environ.get("GITHUB_STEP_SUMMARY"):  # GitHub Actions shows this file on the job page
>         with open(summary, "a") as fh:
>             fh.write(report)
>     print("\n" + report)
>     return 1 if policy.overall(verdicts) == "fail" else 0
> ```

`run` starts a server for each bundled variant you named, or uses the URLs you passed. For each page and each viewport it screenshots the head, asks the six questions, and judges the answers. With a baseline it also screenshots the baseline, sends the pair, and judges that. Every screenshot is saved in `out/` under the variant or hostname, the page, and the viewport, and every verdict is printed as it happens with the input tokens and the request id.

After the loop, the script writes `out/report.md` and `out/results.json`, and appends the report to `GITHUB_STEP_SUMMARY` when that variable is set. API, browser, and image-budget errors exit 2, so a failed check is distinct from a detected visual defect.

### Add the command line and entry point

> **Command line and entry point**
>
> ```python theme={null}
> def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
>     parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
>     head = parser.add_mutually_exclusive_group(required=True)
>     head.add_argument("--variant", choices=VARIANTS, help="serve the bundled site with this stylesheet and check it")
>     head.add_argument("--url", help="check a site that is already running, such as a preview deployment")
>     base = parser.add_mutually_exclusive_group()
>     base.add_argument("--baseline", choices=VARIANTS, help="also compare each screenshot with this bundled variant")
>     base.add_argument("--baseline-url", help="also compare each screenshot with this running site, such as production")
>     parser.add_argument("--pages", nargs="+", default=DEFAULT_PAGES, help="paths to check (default: / and /pricing.html)")
>     parser.add_argument("--out", default="out", help="folder for screenshots, report.md, and results.json")
>     return parser.parse_args(argv)
>
>
> def main() -> int:
>     api_key = os.environ.get("PERPLEXITY_API_KEY")
>     if not api_key:
>         print("Set PERPLEXITY_API_KEY", file=sys.stderr)
>         return 2
>     return run(parse_args(), api_key)
>
>
> if __name__ == "__main__":
>     sys.exit(main())
> ```

`--variant` serves a bundled variant; `--url` checks a site that is already running. `--baseline` and `--baseline-url` add the before-and-after pairs. `--pages` defaults to the two sample pages. `main` refuses to start without `PERPLEXITY_API_KEY`.

## Run the gate

Run the broken variant against `main`, check `main` by itself, then compare the deliberate redesign with `main`. These commands make 20 successful inference requests without retries.

```bash theme={null}
python visual_qa.py --variant pr --baseline main --out out-pr-final
# expected exit code: 1
python visual_qa.py --variant main --out out-main
# expected exit code: 0
python visual_qa.py --variant redesign --baseline main --out out-redesign
# expected exit code: 0
```

The following results were recorded on October 21, 2026 on Linux with Python 3.12 and Playwright Chromium, using code extracted from this page. They consumed 25,974 input tokens across 20 requests. The first command also appended its report to the file named by `GITHUB_STEP_SUMMARY`.

| Run | Requests | Input tokens | Pass | Warn | Fail | Exit code |
| - | -: | -: | -: | -: | -: | -: |
| Broken pull request against main | 8 | 10,568 | 6 | 0 | 2 | 1 |
| Main alone | 4 | 4,310 | 4 | 0 | 0 | 0 |
| Redesign against main | 8 | 11,096 | 8 | 0 | 0 | 0 |

> **Broken variant: recorded report**
>
> ```text theme={null}
> ## Visual QA: FAIL
>
> Checked `pr` against baseline `main` with `pplx-decider-v1.2`: 8 verdicts, 10,568 input tokens.
>
> | Screenshot | Result | Rule |
> |---|---|---|
> | index at 1280 px | pass | all clear |
> | index at 1280 px (before/after) | pass | all clear |
> | index at 390 px | pass | all clear |
> | index at 390 px (before/after) | pass | all clear |
> | pricing at 1280 px | pass | all clear |
> | pricing at 1280 px (before/after) | pass | all clear |
> | pricing at 390 px | fail | text_clipped 96% is at or above 80% |
> | pricing at 390 px (before/after) | fail | content_lost 96% with intentional 32%: looks like a regression |
>
> <details><summary>Probabilities</summary>
>
> - **index at 1280 px**: text_clipped 0%, elements_overlap 0%, cta_visible 100%, low_contrast 8%, layout_broken 1%, error_state 0% (out-pr-final/pr-index-desktop.png)
> - **index at 1280 px (before/after)**: content_lost 1%, intentional 39% (out-pr-final/pr-index-desktop.png)
> - **index at 390 px**: text_clipped 1%, elements_overlap 0%, cta_visible 100%, low_contrast 6%, layout_broken 1%, error_state 0% (out-pr-final/pr-index-mobile.png)
> - **index at 390 px (before/after)**: content_lost 1%, intentional 65% (out-pr-final/pr-index-mobile.png)
> - **pricing at 1280 px**: text_clipped 1%, elements_overlap 0%, cta_visible 100%, low_contrast 9%, layout_broken 1%, error_state 1% (out-pr-final/pr-pricing-desktop.png)
> - **pricing at 1280 px (before/after)**: content_lost 1%, intentional 40% (out-pr-final/pr-pricing-desktop.png)
> - **pricing at 390 px**: text_clipped 96%, elements_overlap 89%, cta_visible 39%, low_contrast 5%, layout_broken 86%, error_state 14% (out-pr-final/pr-pricing-mobile.png)
> - **pricing at 390 px (before/after)**: content_lost 96%, intentional 32% (out-pr-final/pr-pricing-mobile.png)
>
> </details>
> ```

> **Main: recorded report**
>
> ```text theme={null}
> ## Visual QA: PASS
>
> Checked `main` with `pplx-decider-v1.2`: 4 verdicts, 4,310 input tokens.
>
> | Screenshot | Result | Rule |
> |---|---|---|
> | index at 1280 px | pass | all clear |
> | index at 390 px | pass | all clear |
> | pricing at 1280 px | pass | all clear |
> | pricing at 390 px | pass | all clear |
>
> <details><summary>Probabilities</summary>
>
> - **index at 1280 px**: text_clipped 0%, elements_overlap 0%, cta_visible 100%, low_contrast 7%, layout_broken 0%, error_state 1% (out-main/main-index-desktop.png)
> - **index at 390 px**: text_clipped 1%, elements_overlap 0%, cta_visible 100%, low_contrast 12%, layout_broken 1%, error_state 1% (out-main/main-index-mobile.png)
> - **pricing at 1280 px**: text_clipped 1%, elements_overlap 0%, cta_visible 100%, low_contrast 11%, layout_broken 1%, error_state 1% (out-main/main-pricing-desktop.png)
> - **pricing at 390 px**: text_clipped 3%, elements_overlap 2%, cta_visible 100%, low_contrast 9%, layout_broken 1%, error_state 1% (out-main/main-pricing-mobile.png)
>
> </details>
> ```

> **Redesign: recorded report**
>
> ```text theme={null}
> ## Visual QA: PASS
>
> Checked `redesign` against baseline `main` with `pplx-decider-v1.2`: 8 verdicts, 11,096 input tokens.
>
> | Screenshot | Result | Rule |
> |---|---|---|
> | index at 1280 px | pass | all clear |
> | index at 1280 px (before/after) | pass | all clear |
> | index at 390 px | pass | all clear |
> | index at 390 px (before/after) | pass | all clear |
> | pricing at 1280 px | pass | all clear |
> | pricing at 1280 px (before/after) | pass | all clear |
> | pricing at 390 px | pass | all clear |
> | pricing at 390 px (before/after) | pass | all clear |
>
> <details><summary>Probabilities</summary>
>
> - **index at 1280 px**: text_clipped 1%, elements_overlap 0%, cta_visible 100%, low_contrast 7%, layout_broken 0%, error_state 1% (out-redesign/redesign-index-desktop.png)
> - **index at 1280 px (before/after)**: content_lost 0%, intentional 99% (out-redesign/redesign-index-desktop.png)
> - **index at 390 px**: text_clipped 1%, elements_overlap 0%, cta_visible 100%, low_contrast 13%, layout_broken 2%, error_state 1% (out-redesign/redesign-index-mobile.png)
> - **index at 390 px (before/after)**: content_lost 0%, intentional 99% (out-redesign/redesign-index-mobile.png)
> - **pricing at 1280 px**: text_clipped 1%, elements_overlap 0%, cta_visible 100%, low_contrast 8%, layout_broken 1%, error_state 1% (out-redesign/redesign-pricing-desktop.png)
> - **pricing at 1280 px (before/after)**: content_lost 0%, intentional 99% (out-redesign/redesign-pricing-desktop.png)
> - **pricing at 390 px**: text_clipped 1%, elements_overlap 1%, cta_visible 100%, low_contrast 6%, layout_broken 1%, error_state 1% (out-redesign/redesign-pricing-mobile.png)
> - **pricing at 390 px (before/after)**: content_lost 3%, intentional 96% (out-redesign/redesign-pricing-mobile.png)
>
> </details>
> ```

## Read the numbers

The model returned probabilities. `policy.py` produced the verdicts, and `visual_qa.py` set the exit code.

* **Broken mobile pricing page:** `text_clipped` was 96%, `elements_overlap` 89%, and `layout_broken` 86%. The single-image rule failed on the highest score, `text_clipped`. In the pair, `content_lost` was 96% and `intentional` 32%, so that rule also failed.
* **Unchanged pages:** The other six verdicts in the pull-request run passed. Uncertainty about whether an unchanged pair looks deliberate did not matter because `content_lost` stayed below the warning cutoff.
* **Deliberate redesign:** All eight verdicts passed. The four pairs had `content_lost` between 0% and 3% and `intentional` between 96% and 99%.
* **Limits of this check:** The broken variant changes only one CSS rule in a small fixture. This run does not establish false-positive or missed-defect rates for your site, and inferred intent is not proof of a designer's intent. Add representative clean pages, real regressions, and intentional changes before making the gate a required check.

The synthetic tests below bound missed defects and unnecessary reviews on assumed score distributions. They test your policy, not the model's accuracy. Recheck your examples when you change the model, questions, screenshot method, or cutoffs; probabilities near a cutoff need particular care.

## Add it to GitHub Actions

The workflow installs the dependencies and Chromium, runs the gate, and uploads `out/` as an artifact so the screenshots are a click away from the failed job. The report appears in the job summary because `visual_qa.py` writes to `$GITHUB_STEP_SUMMARY` on its own. Add `PERPLEXITY_API_KEY` as a repository secret first.

Put the recipe files at the repository root for this workflow, or set the steps' working directory and artifact path to your recipe folder. The job skips fork pull requests because they do not receive repository secrets. Run it only on trusted code: do not switch to `pull_request_target` to give untrusted changes a key, and do not send sensitive screenshot content without permission.

> **visual-qa.yml**
>
> ```yaml visual-qa.yml theme={null}
> # Save as .github/workflows/visual-qa.yml. The step summary on the job page shows the report
> # that visual_qa.py writes to $GITHUB_STEP_SUMMARY; the screenshots are uploaded as an artifact.
> name: Visual QA
> on:
>   pull_request:
>
> permissions:
>   contents: read
>
> jobs:
>   visual-qa:
>     if: github.event.pull_request.head.repo.full_name == github.repository
>     runs-on: ubuntu-latest
>     steps:
>       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
>       - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
>         with:
>           python-version: "3.12"
>       - name: Install
>         run: |
>           python -m pip install -r requirements.txt
>           python -m playwright install --with-deps chromium
>       - name: Visual QA gate
>         env:
>           PERPLEXITY_API_KEY: ${{ secrets.PERPLEXITY_API_KEY }}
>         # The bundled site: the pull request's stylesheet against main's. For your own site, build or deploy
>         # a preview first and use --url "$PREVIEW_URL" --baseline-url "$PRODUCTION_URL" instead.
>         run: python visual_qa.py --variant pr --baseline main
>       - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
>         if: always()
>         with:
>           name: visual-qa
>           path: out/
> ```

For your own site, the gate needs two URLs it can reach from the runner: the pull request's preview and the baseline. Build the site in an earlier step and serve it with `python -m http.server`, or use a preview deployment URL from your hosting provider, then run `python visual_qa.py --url "$PREVIEW_URL" --baseline-url "$PRODUCTION_URL" --pages / /pricing`. The exit code fails the job on any `fail` verdict; warnings show in the summary and let the job pass.

## Adapt it to your site

* **Your pages.** Pass `--pages` with the paths that matter: the checkout, the dashboard after login, the empty state. For pages behind a login, add a `context.add_cookies(...)` call in `screenshot` with a session cookie from a secret.
* **Your questions.** Edit `SINGLE_QUESTIONS`. A question about a broken image, a missing logo, or an untranslated string is one more `noul` in the dictionary and one more entry in `DEFECT_QUESTIONS`, with no change to the loop. Keep the question general and let the screenshot carry the specifics.
* **A stricter gate.** Lower `FAIL_AT` to 0.7 and the synthetic test will tell you what that costs in clean screenshots failed. Rerun `pytest` after any change to the cutoffs.
* **More viewports.** Add entries to `VIEWPORTS`. Keep the image within the tile budget enforced by `max_height`; oversized pages stop the run rather than receive a partial check.
* **A real baseline.** In CI the baseline is a deployed URL or a build of `main`. Both work with `--baseline-url`.

## Troubleshoot common problems

* `Set PERPLEXITY_API_KEY` means the variable is not set in this terminal. Run the `export` line again.
* `page exceeds the image budget` means the page needs section-by-section capture. The script exits 2 without claiming the page passed.
* `Decisions API failed: 401 {"error":{"message":"Invalid API key provided. Ensure your API key is correct and active.","type":"invalid_api_key","code":401}}` means the key is wrong or inactive.
* `Decisions API failed: 400` is a problem with the request; `error.message` gives the reason. The pair request sends two images per question, well under the limit of eight.
* A request that waits about a minute and then fails with 504 means an image went over 2,048 tiles. If you changed `VIEWPORTS` or `MAX_TILES`, check `max_height`.
* `Decisions API failed: 429` means your organization exceeded the request limit (10 per second) or the token limit, and two retries did not clear it. Another process is likely sharing the limit; wait and rerun.
* `playwright._impl._errors.Error: Executable doesn't exist` means Chromium is not installed for this Playwright version. Run `python -m playwright install chromium` again; in CI, use `--with-deps`.
* `ModuleNotFoundError: No module named 'decisions_client'` means a file is not next to `visual_qa.py`.
* The gate passes a page that looks broken to you. Open `out/results.json` and look at the probabilities. If the right question is between 40% and 80%, it warned and the summary has the screenshot path; if it is below 40%, rephrase the question to describe what you see and rerun.
* The gate warns on every run of a clean page. One question sits high at rest on your design. In the recorded runs, `low_contrast` ran from 6% to 13% on clean screenshots because the gray body text is light; on a site with lighter text it sits higher. Raise `WARN_AT`, give that question its own cutoff in `judge_single`, or rephrase it, then rerun `pytest`.

## Test the pieces

The tests use no API key, network, or browser. Save `test_visual_qa.py` next to the other files and run `pytest`.

> **test\_visual\_qa.py**
>
> ```python test_visual_qa.py theme={null}
> """Offline tests for the policy, the report, and the requests. No API key, network, or browser needed."""
>
> from __future__ import annotations
>
> import json
> import random
> import urllib.request
> from pathlib import Path
> from typing import Any
>
> import httpx
> import pytest
>
> import decisions_client as dc
> import policy
> import visual_qa as vq
>
> CLEAN = {"text_clipped": 0.03, "elements_overlap": 0.02, "cta_visible": 0.97, "low_contrast": 0.01, "layout_broken": 0.04, "error_state": 0.0}
>
>
> def test_policy_on_a_synthetic_distribution_bounds_misses_and_false_alarms() -> None:
>     """Spread risk across outcomes: 85% clean screenshots with a noisy tail, 15% defects of which a quarter are subtle.
>     The gate must miss few defects and flag few clean screenshots. Review-everything misses nothing but flags everything."""
>     rng = random.Random(7)
>     shots: list[tuple[bool, dict[str, float]]] = []
>     for _ in range(20_000):
>         p = {q: rng.betavariate(1, 20) if rng.random() < 0.9 else rng.betavariate(2, 6) for q in policy.DEFECT_QUESTIONS}
>         p["cta_visible"] = rng.betavariate(20, 1)
>         defect = rng.random() < 0.15
>         if defect:
>             q = rng.choice(policy.DEFECT_QUESTIONS + ["cta_visible"])
>             v = rng.betavariate(9, 1.5) if rng.random() < 0.75 else rng.betavariate(3, 3)  # obvious, or subtle and near 50%
>             p[q] = 1 - v if q == "cta_visible" else v
>         shots.append((defect, p))
>     outcomes = [(defect, policy.judge_single("x", p).outcome) for defect, p in shots]
>     defects = [o for d, o in outcomes if d]
>     clean = [o for d, o in outcomes if not d]
>     missed = defects.count("pass") / len(defects)
>     clean_failed = clean.count("fail") / len(clean)
>     clean_flagged = 1 - clean.count("pass") / len(clean)
>     flagged = 1 - [o for _, o in outcomes].count("pass") / len(outcomes)
>     assert missed <= 0.10, f"missed {missed:.1%} of defects"
>     assert clean_failed <= 0.01, f"failed {clean_failed:.1%} of clean screenshots"
>     assert clean_flagged <= 0.10, f"flagged {clean_flagged:.1%} of clean screenshots"
>     assert flagged < 0.25  # review-everything flags 100%; the gate has to do much better to be worth running
>
>
> def test_pair_policy_on_a_synthetic_distribution() -> None:
>     """10% regressions (content lost, not deliberate), 15% redesigns (deliberate, some remove content), 75% unchanged."""
>     rng = random.Random(11)
>     results: dict[str, list[str]] = {"regression": [], "redesign": [], "same": []}
>     for _ in range(20_000):
>         kind = rng.random()
>         if kind < 0.10:
>             p, bucket = {"content_lost": rng.betavariate(9, 1.5), "intentional": rng.betavariate(1.5, 9)}, "regression"
>         elif kind < 0.25:
>             lost = rng.betavariate(6, 3) if rng.random() < 0.3 else rng.betavariate(1.5, 9)
>             p, bucket = {"content_lost": lost, "intentional": rng.betavariate(9, 1.5)}, "redesign"
>         else:
>             p, bucket = {"content_lost": rng.betavariate(1, 20), "intentional": rng.random()}, "same"
>         results[bucket].append(policy.judge_pair("x", p).outcome)
>     assert results["regression"].count("pass") / len(results["regression"]) <= 0.10
>     assert results["redesign"].count("fail") / len(results["redesign"]) <= 0.01
>     assert 1 - results["same"].count("pass") / len(results["same"]) <= 0.10
>
>
> def test_judge_single_fails_warns_and_passes_at_the_cutoffs() -> None:
>     assert policy.judge_single("x", CLEAN).outcome == "pass"
>     assert policy.judge_single("x", {**CLEAN, "elements_overlap": 0.80}).rule.startswith("elements_overlap 80% is at or above 80%")
>     assert policy.judge_single("x", {**CLEAN, "text_clipped": 0.79}).outcome == "warn"
>     assert policy.judge_single("x", {**CLEAN, "text_clipped": 0.39}).outcome == "pass"
>     assert policy.judge_single("x", {**CLEAN, "cta_visible": 0.20}).outcome == "fail"
>     assert policy.judge_single("x", {**CLEAN, "cta_visible": 0.55}).outcome == "warn"
>     worst = policy.judge_single("x", {**CLEAN, "layout_broken": 0.85, "text_clipped": 0.95})
>     assert worst.rule.startswith("text_clipped 95%")  # the worst question names the rule
>
>
> def test_judge_pair_fails_regressions_and_only_warns_on_deliberate_removals() -> None:
>     assert policy.judge_pair("x", {"content_lost": 0.92, "intentional": 0.08}).outcome == "fail"
>     assert policy.judge_pair("x", {"content_lost": 0.92, "intentional": 0.85}).outcome == "warn"
>     assert policy.judge_pair("x", {"content_lost": 0.45, "intentional": 0.10}).outcome == "warn"
>     assert policy.judge_pair("x", {"content_lost": 0.05, "intentional": 0.95}).outcome == "pass"
>
>
> def test_overall_is_the_worst_verdict() -> None:
>     verdicts = [policy.Verdict("a", "pass", ""), policy.Verdict("b", "warn", ""), policy.Verdict("c", "fail", "")]
>     assert policy.overall(verdicts) == "fail"
>     assert policy.overall(verdicts[:2]) == "warn"
>     assert policy.overall([]) == "pass"
>
>
> def test_render_report_lists_every_verdict_with_its_rule_and_probabilities() -> None:
>     verdicts = [
>         policy.judge_single("pricing at 390 px", {**CLEAN, "elements_overlap": 0.91}, "out/pr-pricing-mobile.png"),
>         policy.judge_pair("pricing at 390 px (before/after)", {"content_lost": 0.9, "intentional": 0.1}),
>     ]
>     report = policy.render_report(verdicts, "pr", "main", 12_345, "pplx-decider-v1.2")
>     assert report.startswith("## Visual QA: FAIL")
>     assert "Checked `pr` against baseline `main` with `pplx-decider-v1.2`: 2 verdicts, 12,345 input tokens." in report
>     assert "| pricing at 390 px | fail | elements_overlap 91% is at or above 80% |" in report
>     assert "- **pricing at 390 px**: text_clipped 3%, elements_overlap 91%" in report
>     assert "(out/pr-pricing-mobile.png)" in report
>     assert "intentional 10%" in report
>
>
> def test_single_body_carries_a_caption_the_image_and_six_questions() -> None:
>     body = dc.single_body(b"png-bytes", "pricing at 390 px")
>     assert body["model"] == "pplx-decider-v1.2"
>     assert body["state"][0] == "Screenshot of pricing at 390 px."
>     assert body["state"][1]["type"] == "image_url"
>     assert body["state"][1]["image_url"]["url"] == "data:image/png;base64,cG5nLWJ5dGVz"
>     assert list(body["questions"]) == ["text_clipped", "elements_overlap", "cta_visible", "low_contrast", "layout_broken", "error_state"]
>     assert all(q["type"] == "noul" for q in body["questions"].values())
>
>
> def test_pair_body_puts_before_then_after_each_behind_its_caption() -> None:
>     body = dc.pair_body(b"old", b"new", "index at 1280 px")
>     kinds = [item if isinstance(item, str) else item["type"] for item in body["state"]]
>     assert kinds == ["Two screenshots of index at 1280 px.", "BEFORE (baseline):", "image_url", "AFTER (change under review):", "image_url"]
>     assert body["state"][2]["image_url"]["url"].endswith("b2xk")  # "old"
>     assert body["state"][4]["image_url"]["url"].endswith("bmV3")  # "new"
>     assert list(body["questions"]) == ["content_lost", "intentional"]
>
>
> def fake_client(responses: list[httpx.Response], calls: list[dict[str, Any]]) -> httpx.Client:
>     """An httpx.Client that answers with the given responses in order and records each request body in calls."""
>
>     def handler(request: httpx.Request) -> httpx.Response:
>         calls.append(json.loads(request.content))
>         return responses[len(calls) - 1]
>
>     return httpx.Client(transport=httpx.MockTransport(handler))
>
>
> OK: dict[str, Any] = {
>     "model": "pplx-decider-v1.2",
>     "answers": {name: {"type": "noul", "noul": p} for name, p in CLEAN.items()},
>     "usage": {"input_tokens": 1234, "output_tokens": 6},
> }
>
>
> def test_ask_returns_probabilities_tokens_and_request_id_and_retries_a_429() -> None:
>     calls: list[dict[str, Any]] = []
>     responses = [
>         httpx.Response(429, json={"error": "slow down"}, headers={"retry-after": "0"}),
>         httpx.Response(200, json=OK, headers={"x-request-id": "rid-1"}),
>     ]
>     with fake_client(responses, calls) as client:
>         probs, tokens, rid = dc.ask(client, dc.single_body(b"png", "x"))
>     assert (probs, tokens, rid, len(calls)) == (CLEAN, 1234, "rid-1", 2)
>     assert calls[0] == calls[1]
>
>
> def test_ask_raises_on_a_400_and_on_a_200_missing_an_answer() -> None:
>     calls: list[dict[str, Any]] = []
>     with fake_client([httpx.Response(400, json={"error": {"message": "bad state"}})], calls) as client:
>         try:
>             dc.ask(client, dc.single_body(b"png", "x"))
>             raise AssertionError("expected DecisionsError")
>         except dc.DecisionsError as exc:
>             assert "400" in str(exc) and "bad state" in str(exc)
>     partial = {**OK, "answers": {k: v for k, v in OK["answers"].items() if k != "error_state"}}
>     with fake_client([httpx.Response(200, json=partial)], []) as client:
>         try:
>             dc.ask(client, dc.single_body(b"png", "x"))
>             raise AssertionError("expected DecisionsError")
>         except dc.DecisionsError as exc:
>             assert "cannot read" in str(exc)
>
>
> def test_max_height_keeps_every_width_inside_the_tile_budget() -> None:
>     for width in (390, 1280, 1440, 2048):
>         assert round(width / 32) * (vq.max_height(width) // 32) <= vq.MAX_TILES
>     assert vq.max_height(1280) == 1632
>     assert vq.slug("/pricing.html") == "pricing" and vq.slug("/") == "index"
>
>
> @pytest.mark.parametrize("invalid", [float("nan"), float("inf"), -0.1, 1.1])
> def test_invalid_probabilities_cannot_pass_the_gate(invalid: float) -> None:
>     body = {**OK, "answers": {**OK["answers"], "text_clipped": {"noul": invalid}}}
>     response = httpx.Response(200, text=json.dumps(body))
>     with fake_client([response], []) as client, pytest.raises(dc.DecisionsError, match="cannot read"):
>         dc.ask(client, dc.single_body(b"png", "x"))
>
>
> def test_oversized_page_is_not_silently_cropped() -> None:
>     from unittest.mock import MagicMock
>
>     browser = MagicMock()
>     page = browser.new_context.return_value.new_page.return_value
>     page.evaluate.return_value = 5000
>     with pytest.raises(ValueError, match="split it into sections"):
>         vq.screenshot(browser, "http://localhost/", 1280, 800)
>     page.screenshot.assert_not_called()
>     browser.new_context.return_value.close.assert_called_once()
>
>
> def test_screenshot_preserves_viewport_width_and_full_height() -> None:
>     from unittest.mock import MagicMock
>
>     browser = MagicMock()
>     page = browser.new_context.return_value.new_page.return_value
>     page.evaluate.return_value = 1200
>     vq.screenshot(browser, "http://localhost/", 390, 844)
>     page.screenshot.assert_called_once_with(full_page=True, clip={"x": 0, "y": 0, "width": 390, "height": 1200}, animations="disabled")
>
>
> def test_run_appends_report_to_job_summary(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
>     from unittest.mock import MagicMock
>
>     monkeypatch.setattr(vq, "sync_playwright", MagicMock())
>     monkeypatch.setattr(vq, "screenshot", lambda *args: b"png")
>     monkeypatch.setattr(dc, "ask", lambda *args: (dict(CLEAN), 123, "rid"))
>     summary = tmp_path / "summary.md"
>     summary.write_text("Existing summary\n")
>     monkeypatch.setenv("GITHUB_STEP_SUMMARY", str(summary))
>     args = vq.parse_args(["--url", "https://example.com", "--pages", "/", "--out", str(tmp_path / "out")])
>     assert vq.run(args, "unused-test-key") == 0
>     report = (tmp_path / "out" / "report.md").read_text()
>     assert summary.read_text() == "Existing summary\n" + report
>
>
> def test_site_server_serves_the_variant_as_theme_css() -> None:
>     server, url = vq.serve("pr")
>     try:
>         theme = urllib.request.urlopen(f"{url}/theme.css", timeout=5).read().decode()
>         page = urllib.request.urlopen(f"{url}/pricing.html", timeout=5).read().decode()
>     finally:
>         server.shutdown()
>     assert theme == (vq.SITE_DIR / "pr.css").read_text()
>     assert '<link rel="stylesheet" href="/theme.css">' in page
> ```

The first two tests are the synthetic distributions from [Choose the cutoffs before writing code](#choose-the-cutoffs-before-writing-code). The rest check the cutoffs at their edges, the report text, the shape of both request bodies (caption, image part, question names, BEFORE then AFTER), retry on 429, errors on 400 and on an unreadable 200, the tile budget, and the local server's variant mapping.

```text theme={null}
...................                                                      [100%]
19 passed
```

## Copy the complete files

The exact files used for the runs above.

> **requirements.txt**
>
> ```text requirements.txt theme={null}
> httpx==0.28.1
> playwright==1.64.0
> pytest==9.1.1
> ```

> **pyproject.toml**
>
> ```toml pyproject.toml theme={null}
> [tool.ruff]
> line-length = 150
> target-version = "py310"
>
> [tool.ruff.lint]
> select = ["E", "F", "I", "B", "UP", "C4", "SIM"]
>
> [tool.mypy]
> strict = true
> ignore_missing_imports = true
>
> [tool.pytest.ini_options]
> addopts = "-q"
> ```

> **decisions\_client.py**
>
> ```python decisions_client.py theme={null}
> """The part of the gate that talks to the Decisions API: the questions, the request bodies, and
> bounded retries. Nothing in this file knows about browsers, pages, or cutoffs."""
>
> from __future__ import annotations
>
> import base64
> import time
> from typing import Any
>
> import httpx
>
> API_URL = "https://api.perplexity.ai/v1/decisions"
> MODEL = "pplx-decider-v1.2"  # groups the questions and bills the screenshot once
> REQUEST_TIMEOUT_S = 90.0  # a 1,000-tile screenshot answers in about a second; this covers a slow one
> RETRY_STATUSES = {429, 500, 502, 503, 504}
> MAX_RETRIES = 2  # two more tries for those and for dropped connections, waiting Retry-After seconds (or 1 s, then 2 s)
> MAX_RETRY_DELAY_S = 30.0  # if Retry-After asks for more than this, stop with an error instead of waiting
>
> # Six yes-or-no questions about one screenshot. The answer to each is the probability of yes.
> SINGLE_QUESTIONS: dict[str, dict[str, Any]] = {
>     "text_clipped": {"type": "noul", "instructions": "Is any text clipped, cut off mid-word, or drawn on top of other text?"},
>     "elements_overlap": {
>         "type": "noul",
>         "instructions": "Do any elements such as cards, images, or buttons overlap or cover other content that should be visible?",
>     },
>     "cta_visible": {
>         "type": "noul",
>         "instructions": "Is a primary call-to-action button, such as Start free trial, Choose a plan, Get started, or Buy, "
>         "fully visible and readable?",
>     },
>     "low_contrast": {"type": "noul", "instructions": "Is any text hard to read because it has too little contrast with its background?"},
>     "layout_broken": {
>         "type": "noul",
>         "instructions": "Is the layout broken, with elements pushed off-screen, misaligned, stacked wrongly, or a horizontal scrollbar?",
>     },
>     "error_state": {"type": "noul", "instructions": "Does the page look like a loading, blank, or error state rather than a finished page?"},
> }
>
> # Two questions about a before-and-after pair. BEFORE is the baseline, AFTER is the change under review.
> PAIR_QUESTIONS: dict[str, dict[str, Any]] = {
>     "content_lost": {
>         "type": "noul",
>         "instructions": "Is content that is present in the BEFORE screenshot (text, buttons, or whole sections) missing or hidden "
>         "in the AFTER screenshot?",
>     },
>     "intentional": {
>         "type": "noul",
>         "instructions": "Do the differences between BEFORE and AFTER look like a deliberate design change rather than a rendering bug?",
>     },
> }
>
>
> class DecisionsError(Exception):
>     """The Decisions API returned an error response, or a response the script cannot read."""
>
>
> def make_client(api_key: str) -> httpx.Client:
>     """One HTTP client for the whole run: it carries the key, the timeout, and a reused connection."""
>     return httpx.Client(headers={"Authorization": f"Bearer {api_key}"}, timeout=REQUEST_TIMEOUT_S)
>
>
> def image_part(png: bytes) -> dict[str, Any]:
>     """One screenshot as the OpenAI-style image part the Decisions API reads inside state."""
>     return {"type": "image_url", "image_url": {"url": "data:image/png;base64," + base64.b64encode(png).decode()}}
>
>
> def single_body(png: bytes, label: str) -> dict[str, Any]:
>     """The request for one screenshot: a caption, the image, and the six questions."""
>     return {"model": MODEL, "state": [f"Screenshot of {label}.", image_part(png)], "questions": SINGLE_QUESTIONS}
>
>
> def pair_body(before: bytes, after: bytes, label: str) -> dict[str, Any]:
>     """The request for a before-and-after pair. Order matters: the model reads state top to bottom,
>     so each caption comes right before its image."""
>     return {
>         "model": MODEL,
>         "state": [f"Two screenshots of {label}.", "BEFORE (baseline):", image_part(before), "AFTER (change under review):", image_part(after)],
>         "questions": PAIR_QUESTIONS,
>     }
>
>
> def ask(client: httpx.Client, body: dict[str, Any]) -> tuple[dict[str, float], int, str | None]:
>     """Send one request and return the probability of yes per question, the input tokens, and the request id.
>     Retries 429, 5xx, and dropped connections a bounded number of times; raises DecisionsError on any other failure."""
>     for attempt in range(MAX_RETRIES + 1):
>         try:
>             response = client.post(API_URL, json=body)
>         except httpx.TransportError:  # dropped connection or read timeout
>             if attempt == MAX_RETRIES:
>                 raise
>             time.sleep(float(2**attempt))
>             continue
>         rid = response.headers.get("x-request-id")
>         if response.status_code == 200:
>             return _parse(response, body["questions"], rid)
>         if response.status_code not in RETRY_STATUSES or attempt == MAX_RETRIES:
>             break
>         delay = _retry_delay(response, attempt)
>         if delay > MAX_RETRY_DELAY_S:
>             raise DecisionsError(f"Decisions API asked to wait {delay:.0f} s before retrying ({response.status_code}); stopping instead")
>         time.sleep(delay)
>     raise DecisionsError(f"Decisions API failed: {response.status_code} {response.text.strip()}" + (f" (x-request-id {rid})" if rid else ""))
>
>
> def _parse(response: httpx.Response, questions: dict[str, Any], rid: str | None) -> tuple[dict[str, float], int, str | None]:
>     """The probabilities out of a 200, checked for every question this script asked. Anything else is a DecisionsError."""
>     try:
>         data = response.json()
>         probabilities = {name: float(data["answers"][name]["noul"]) for name in questions}
>         if any(not 0 <= p <= 1 for p in probabilities.values()):
>             raise ValueError("probabilities must be finite numbers from 0 to 1")
>         return probabilities, int(data["usage"]["input_tokens"]), rid
>     except (ValueError, KeyError, TypeError) as exc:
>         raise DecisionsError(
>             f"Decisions API returned a 200 this script cannot read ({exc!r}): {response.text[:200]!r}" + (f" (x-request-id {rid})" if rid else "")
>         ) from exc
>
>
> def _retry_delay(response: httpx.Response, attempt: int) -> float:
>     """Seconds to wait before retrying: the Retry-After header if it is a number, else 1 s then 2 s."""
>     try:
>         return max(0.0, float(response.headers.get("retry-after", "")))
>     except ValueError:
>         return float(2**attempt)
> ```

> **policy.py**
>
> ```python policy.py theme={null}
> """The gate's policy and report. The model returns probabilities; this file compares them to cutoffs,
> names the rule that fired, and renders the Markdown that lands in the GitHub Actions job summary."""
>
> from __future__ import annotations
>
> from dataclasses import dataclass, field
>
> FAIL_AT = 0.80  # a defect question at or above this fails the build
> WARN_AT = 0.40  # at or above this (and below FAIL_AT) the build passes with a warning to look at the screenshot
> DEFECT_QUESTIONS = ["text_clipped", "elements_overlap", "low_contrast", "layout_broken", "error_state"]
> # cta_visible is the one question where a high probability is good, so its cutoffs are mirrored.
> CTA_FAIL_BELOW = round(1 - FAIL_AT, 2)
> CTA_WARN_BELOW = round(1 - WARN_AT, 2)
>
> ORDER = {"fail": 0, "warn": 1, "pass": 2}
>
>
> @dataclass
> class Verdict:
>     """One gate result: the screenshot it is about, the outcome, the rule that fired, and every probability."""
>
>     label: str  # "pricing.html at 390 px" or "pricing.html at 390 px (before/after)"
>     outcome: str  # pass, warn, or fail
>     rule: str  # the rule that fired, or "all clear"
>     probabilities: dict[str, float] = field(default_factory=dict)
>     screenshot: str = ""  # path of the AFTER screenshot, for the report
>
>
> def judge_single(label: str, p: dict[str, float], screenshot: str = "") -> Verdict:
>     """Compare the six probabilities for one screenshot to the cutoffs. The worst defect question wins."""
>     worst = max(DEFECT_QUESTIONS, key=lambda q: p[q])
>     if p[worst] >= FAIL_AT:
>         return Verdict(label, "fail", f"{worst} {p[worst]:.0%} is at or above {FAIL_AT:.0%}", p, screenshot)
>     if p["cta_visible"] <= CTA_FAIL_BELOW:
>         return Verdict(label, "fail", f"cta_visible {p['cta_visible']:.0%} is at or below {CTA_FAIL_BELOW:.0%}", p, screenshot)
>     if p[worst] >= WARN_AT:
>         return Verdict(label, "warn", f"{worst} {p[worst]:.0%} is at or above {WARN_AT:.0%}; look at the screenshot", p, screenshot)
>     if p["cta_visible"] <= CTA_WARN_BELOW:
>         rule = f"cta_visible {p['cta_visible']:.0%} is at or below {CTA_WARN_BELOW:.0%}; look at the screenshot"
>         return Verdict(label, "warn", rule, p, screenshot)
>     return Verdict(label, "pass", "all clear", p, screenshot)
>
>
> def judge_pair(label: str, p: dict[str, float], screenshot: str = "") -> Verdict:
>     """Compare the two before-and-after probabilities to the cutoffs. Lost content fails only when the
>     change does not look deliberate; a deliberate removal is flagged for a person to confirm."""
>     lost, intentional = p["content_lost"], p["intentional"]
>     if lost >= FAIL_AT and intentional <= WARN_AT:
>         return Verdict(label, "fail", f"content_lost {lost:.0%} with intentional {intentional:.0%}: looks like a regression", p, screenshot)
>     if lost >= WARN_AT:
>         return Verdict(label, "warn", f"content_lost {lost:.0%} with intentional {intentional:.0%}; confirm the removal was meant", p, screenshot)
>     return Verdict(label, "pass", "all clear", p, screenshot)
>
>
> def overall(verdicts: list[Verdict]) -> str:
>     """The run's outcome is the worst verdict in it."""
>     return min((v.outcome for v in verdicts), key=lambda outcome: ORDER[outcome], default="pass")
>
>
> def render_report(verdicts: list[Verdict], head: str, base: str | None, tokens: int, model: str) -> str:
>     """GitHub-flavored Markdown: a headline, one table row per verdict, and the probabilities behind each row."""
>     result = overall(verdicts)
>     lines = [f"## Visual QA: {result.upper()}", ""]
>     against = f" against baseline `{base}`" if base else ""
>     lines.append(f"Checked `{head}`{against} with `{model}`: {len(verdicts)} verdicts, {tokens:,} input tokens.")
>     lines += ["", "| Screenshot | Result | Rule |", "|---|---|---|"]
>     for v in verdicts:
>         lines.append(f"| {v.label} | {v.outcome} | {v.rule} |")
>     lines += ["", "<details><summary>Probabilities</summary>", ""]
>     for v in verdicts:
>         probs = ", ".join(f"{k} {p:.0%}" for k, p in v.probabilities.items())
>         lines.append(f"- **{v.label}**: {probs}" + (f" ({v.screenshot})" if v.screenshot else ""))
>     lines += ["", "</details>", ""]
>     return "\n".join(lines)
> ```

> **visual\_qa.py**
>
> ```python visual_qa.py theme={null}
> """A visual QA gate for CI. Screenshots each page at two widths, sends every screenshot to the Decisions API
> with six yes-or-no questions, and (with a baseline) sends before-and-after pairs with two more. The model
> returns probabilities; policy.py compares them to cutoffs. The exit code is 1 when any verdict is fail.
>
>     python visual_qa.py --variant pr --baseline main        # the bundled site: a pull request against main
>     python visual_qa.py --url https://preview.example.com --baseline-url https://www.example.com
> """
>
> from __future__ import annotations
>
> import argparse
> import functools
> import json
> import os
> import sys
> import threading
> from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
> from pathlib import Path
> from typing import Any
> from urllib.parse import urlparse
>
> import httpx
> from playwright.sync_api import Browser, FloatRect, sync_playwright
> from playwright.sync_api import Error as PlaywrightError
>
> import decisions_client as dc
> import policy
>
> SITE_DIR = Path(__file__).parent / "site"
> VARIANTS = ["main", "pr", "redesign"]  # each one is site/<variant>.css, served as /theme.css
> VIEWPORTS = {"desktop": (1280, 800), "mobile": (390, 844)}
> MAX_TILES = 2048  # the Decisions API reads images in 32 x 32 tiles; a larger image waits about a minute and returns 504
> DEFAULT_PAGES = ["/", "/pricing.html"]
>
>
> class SiteHandler(SimpleHTTPRequestHandler):
>     """Serves site/ as static files, except /theme.css, which comes from site/<variant>.css."""
>
>     variant = "main"
>
>     def translate_path(self, path: str) -> str:
>         if path.split("?")[0] == "/theme.css":
>             return str(SITE_DIR / f"{self.variant}.css")
>         return super().translate_path(path)
>
>     def log_message(self, format: str, *args: Any) -> None:  # noqa: A002 (the base class names it format)
>         pass  # keep the console for the gate's own output
>
>
> def serve(variant: str) -> tuple[ThreadingHTTPServer, str]:
>     """Start one local server for a variant on a free port. Returns the server and its base URL."""
>     handler = type(f"Handler_{variant}", (SiteHandler,), {"variant": variant})
>     server = ThreadingHTTPServer(("127.0.0.1", 0), functools.partial(handler, directory=str(SITE_DIR)))
>     threading.Thread(target=server.serve_forever, daemon=True).start()
>     return server, f"http://127.0.0.1:{server.server_address[1]}"
>
>
> def max_height(width: int) -> int:
>     """The tallest screenshot of this width that stays within MAX_TILES, as a multiple of 32 pixels."""
>     return (MAX_TILES // max(1, round(width / 32))) * 32
>
>
> def screenshot(browser: Browser, url: str, width: int, height: int) -> bytes:
>     """A full-page PNG. Reject pages over the tile budget rather than silently omit content."""
>     context = browser.new_context(viewport={"width": width, "height": height}, reduced_motion="reduce")
>     try:
>         page = context.new_page()
>         page.goto(url, wait_until="networkidle")
>         page.wait_for_timeout(250)  # web fonts and late layout
>         full = int(page.evaluate("document.documentElement.scrollHeight"))
>         if max(full, height) > max_height(width):
>             raise ValueError(f"{url}: page exceeds the image budget at {width} px; split it into sections before checking")
>         clip: FloatRect = {"x": 0, "y": 0, "width": width, "height": max(full, height)}
>         return page.screenshot(full_page=True, clip=clip, animations="disabled")
>     finally:
>         context.close()
>
>
> def slug(path: str) -> str:
>     """'/pricing.html' -> 'pricing', '/' -> 'index'."""
>     return path.strip("/").removesuffix(".html").replace("/", "-") or "index"
>
>
> def run(args: argparse.Namespace, api_key: str) -> int:
>     """Screenshot, ask, judge, report. Returns the process exit code."""
>     out = Path(args.out)
>     out.mkdir(exist_ok=True)
>     servers: list[ThreadingHTTPServer] = []
>     head_url, base_url = args.url, args.baseline_url
>     head_name = args.variant or urlparse(args.url).hostname or "head"  # names the screenshots and the report
>     base_name = args.baseline or (urlparse(args.baseline_url).hostname if args.baseline_url else None)
>     if args.variant:
>         server, head_url = serve(args.variant)
>         servers.append(server)
>     if args.baseline:
>         server, base_url = serve(args.baseline)
>         servers.append(server)
>
>     verdicts: list[policy.Verdict] = []
>     tokens = 0
>     try:
>         with sync_playwright() as pw, pw.chromium.launch() as browser, dc.make_client(api_key) as client:
>             for path in args.pages:
>                 for view, (width, height) in VIEWPORTS.items():
>                     label = f"{slug(path)} at {width} px"
>                     after = screenshot(browser, head_url + path, width, height)
>                     shot = out / f"{head_name}-{slug(path)}-{view}.png"
>                     shot.write_bytes(after)
>                     probs, used, rid = dc.ask(client, dc.single_body(after, label))
>                     tokens += used
>                     verdicts.append(policy.judge_single(label, probs, str(shot)))
>                     print(f"{label}: {verdicts[-1].outcome} ({verdicts[-1].rule}) [{used} tokens, {rid}]")
>                     if base_url:
>                         before = screenshot(browser, base_url + path, width, height)
>                         (out / f"{base_name}-{slug(path)}-{view}.png").write_bytes(before)
>                         probs, used, rid = dc.ask(client, dc.pair_body(before, after, label))
>                         tokens += used
>                         verdicts.append(policy.judge_pair(f"{label} (before/after)", probs, str(shot)))
>                         print(f"{label} (before/after): {verdicts[-1].outcome} ({verdicts[-1].rule}) [{used} tokens, {rid}]")
>     except (dc.DecisionsError, httpx.HTTPError, PlaywrightError, ValueError) as exc:
>         print(f"error: {exc}", file=sys.stderr)
>         return 2
>     finally:
>         for server in servers:
>             server.shutdown()
>
>     report = policy.render_report(verdicts, head_name, base_name, tokens, dc.MODEL)
>     (out / "report.md").write_text(report)
>     (out / "results.json").write_text(json.dumps([vars(v) for v in verdicts], indent=2))
>     if summary := os.environ.get("GITHUB_STEP_SUMMARY"):  # GitHub Actions shows this file on the job page
>         with open(summary, "a") as fh:
>             fh.write(report)
>     print("\n" + report)
>     return 1 if policy.overall(verdicts) == "fail" else 0
>
>
> def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
>     parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
>     head = parser.add_mutually_exclusive_group(required=True)
>     head.add_argument("--variant", choices=VARIANTS, help="serve the bundled site with this stylesheet and check it")
>     head.add_argument("--url", help="check a site that is already running, such as a preview deployment")
>     base = parser.add_mutually_exclusive_group()
>     base.add_argument("--baseline", choices=VARIANTS, help="also compare each screenshot with this bundled variant")
>     base.add_argument("--baseline-url", help="also compare each screenshot with this running site, such as production")
>     parser.add_argument("--pages", nargs="+", default=DEFAULT_PAGES, help="paths to check (default: / and /pricing.html)")
>     parser.add_argument("--out", default="out", help="folder for screenshots, report.md, and results.json")
>     return parser.parse_args(argv)
>
>
> def main() -> int:
>     api_key = os.environ.get("PERPLEXITY_API_KEY")
>     if not api_key:
>         print("Set PERPLEXITY_API_KEY", file=sys.stderr)
>         return 2
>     return run(parse_args(), api_key)
>
>
> if __name__ == "__main__":
>     sys.exit(main())
> ```

> **test\_visual\_qa.py**
>
> ```python test_visual_qa.py theme={null}
> """Offline tests for the policy, the report, and the requests. No API key, network, or browser needed."""
>
> from __future__ import annotations
>
> import json
> import random
> import urllib.request
> from pathlib import Path
> from typing import Any
>
> import httpx
> import pytest
>
> import decisions_client as dc
> import policy
> import visual_qa as vq
>
> CLEAN = {"text_clipped": 0.03, "elements_overlap": 0.02, "cta_visible": 0.97, "low_contrast": 0.01, "layout_broken": 0.04, "error_state": 0.0}
>
>
> def test_policy_on_a_synthetic_distribution_bounds_misses_and_false_alarms() -> None:
>     """Spread risk across outcomes: 85% clean screenshots with a noisy tail, 15% defects of which a quarter are subtle.
>     The gate must miss few defects and flag few clean screenshots. Review-everything misses nothing but flags everything."""
>     rng = random.Random(7)
>     shots: list[tuple[bool, dict[str, float]]] = []
>     for _ in range(20_000):
>         p = {q: rng.betavariate(1, 20) if rng.random() < 0.9 else rng.betavariate(2, 6) for q in policy.DEFECT_QUESTIONS}
>         p["cta_visible"] = rng.betavariate(20, 1)
>         defect = rng.random() < 0.15
>         if defect:
>             q = rng.choice(policy.DEFECT_QUESTIONS + ["cta_visible"])
>             v = rng.betavariate(9, 1.5) if rng.random() < 0.75 else rng.betavariate(3, 3)  # obvious, or subtle and near 50%
>             p[q] = 1 - v if q == "cta_visible" else v
>         shots.append((defect, p))
>     outcomes = [(defect, policy.judge_single("x", p).outcome) for defect, p in shots]
>     defects = [o for d, o in outcomes if d]
>     clean = [o for d, o in outcomes if not d]
>     missed = defects.count("pass") / len(defects)
>     clean_failed = clean.count("fail") / len(clean)
>     clean_flagged = 1 - clean.count("pass") / len(clean)
>     flagged = 1 - [o for _, o in outcomes].count("pass") / len(outcomes)
>     assert missed <= 0.10, f"missed {missed:.1%} of defects"
>     assert clean_failed <= 0.01, f"failed {clean_failed:.1%} of clean screenshots"
>     assert clean_flagged <= 0.10, f"flagged {clean_flagged:.1%} of clean screenshots"
>     assert flagged < 0.25  # review-everything flags 100%; the gate has to do much better to be worth running
>
>
> def test_pair_policy_on_a_synthetic_distribution() -> None:
>     """10% regressions (content lost, not deliberate), 15% redesigns (deliberate, some remove content), 75% unchanged."""
>     rng = random.Random(11)
>     results: dict[str, list[str]] = {"regression": [], "redesign": [], "same": []}
>     for _ in range(20_000):
>         kind = rng.random()
>         if kind < 0.10:
>             p, bucket = {"content_lost": rng.betavariate(9, 1.5), "intentional": rng.betavariate(1.5, 9)}, "regression"
>         elif kind < 0.25:
>             lost = rng.betavariate(6, 3) if rng.random() < 0.3 else rng.betavariate(1.5, 9)
>             p, bucket = {"content_lost": lost, "intentional": rng.betavariate(9, 1.5)}, "redesign"
>         else:
>             p, bucket = {"content_lost": rng.betavariate(1, 20), "intentional": rng.random()}, "same"
>         results[bucket].append(policy.judge_pair("x", p).outcome)
>     assert results["regression"].count("pass") / len(results["regression"]) <= 0.10
>     assert results["redesign"].count("fail") / len(results["redesign"]) <= 0.01
>     assert 1 - results["same"].count("pass") / len(results["same"]) <= 0.10
>
>
> def test_judge_single_fails_warns_and_passes_at_the_cutoffs() -> None:
>     assert policy.judge_single("x", CLEAN).outcome == "pass"
>     assert policy.judge_single("x", {**CLEAN, "elements_overlap": 0.80}).rule.startswith("elements_overlap 80% is at or above 80%")
>     assert policy.judge_single("x", {**CLEAN, "text_clipped": 0.79}).outcome == "warn"
>     assert policy.judge_single("x", {**CLEAN, "text_clipped": 0.39}).outcome == "pass"
>     assert policy.judge_single("x", {**CLEAN, "cta_visible": 0.20}).outcome == "fail"
>     assert policy.judge_single("x", {**CLEAN, "cta_visible": 0.55}).outcome == "warn"
>     worst = policy.judge_single("x", {**CLEAN, "layout_broken": 0.85, "text_clipped": 0.95})
>     assert worst.rule.startswith("text_clipped 95%")  # the worst question names the rule
>
>
> def test_judge_pair_fails_regressions_and_only_warns_on_deliberate_removals() -> None:
>     assert policy.judge_pair("x", {"content_lost": 0.92, "intentional": 0.08}).outcome == "fail"
>     assert policy.judge_pair("x", {"content_lost": 0.92, "intentional": 0.85}).outcome == "warn"
>     assert policy.judge_pair("x", {"content_lost": 0.45, "intentional": 0.10}).outcome == "warn"
>     assert policy.judge_pair("x", {"content_lost": 0.05, "intentional": 0.95}).outcome == "pass"
>
>
> def test_overall_is_the_worst_verdict() -> None:
>     verdicts = [policy.Verdict("a", "pass", ""), policy.Verdict("b", "warn", ""), policy.Verdict("c", "fail", "")]
>     assert policy.overall(verdicts) == "fail"
>     assert policy.overall(verdicts[:2]) == "warn"
>     assert policy.overall([]) == "pass"
>
>
> def test_render_report_lists_every_verdict_with_its_rule_and_probabilities() -> None:
>     verdicts = [
>         policy.judge_single("pricing at 390 px", {**CLEAN, "elements_overlap": 0.91}, "out/pr-pricing-mobile.png"),
>         policy.judge_pair("pricing at 390 px (before/after)", {"content_lost": 0.9, "intentional": 0.1}),
>     ]
>     report = policy.render_report(verdicts, "pr", "main", 12_345, "pplx-decider-v1.2")
>     assert report.startswith("## Visual QA: FAIL")
>     assert "Checked `pr` against baseline `main` with `pplx-decider-v1.2`: 2 verdicts, 12,345 input tokens." in report
>     assert "| pricing at 390 px | fail | elements_overlap 91% is at or above 80% |" in report
>     assert "- **pricing at 390 px**: text_clipped 3%, elements_overlap 91%" in report
>     assert "(out/pr-pricing-mobile.png)" in report
>     assert "intentional 10%" in report
>
>
> def test_single_body_carries_a_caption_the_image_and_six_questions() -> None:
>     body = dc.single_body(b"png-bytes", "pricing at 390 px")
>     assert body["model"] == "pplx-decider-v1.2"
>     assert body["state"][0] == "Screenshot of pricing at 390 px."
>     assert body["state"][1]["type"] == "image_url"
>     assert body["state"][1]["image_url"]["url"] == "data:image/png;base64,cG5nLWJ5dGVz"
>     assert list(body["questions"]) == ["text_clipped", "elements_overlap", "cta_visible", "low_contrast", "layout_broken", "error_state"]
>     assert all(q["type"] == "noul" for q in body["questions"].values())
>
>
> def test_pair_body_puts_before_then_after_each_behind_its_caption() -> None:
>     body = dc.pair_body(b"old", b"new", "index at 1280 px")
>     kinds = [item if isinstance(item, str) else item["type"] for item in body["state"]]
>     assert kinds == ["Two screenshots of index at 1280 px.", "BEFORE (baseline):", "image_url", "AFTER (change under review):", "image_url"]
>     assert body["state"][2]["image_url"]["url"].endswith("b2xk")  # "old"
>     assert body["state"][4]["image_url"]["url"].endswith("bmV3")  # "new"
>     assert list(body["questions"]) == ["content_lost", "intentional"]
>
>
> def fake_client(responses: list[httpx.Response], calls: list[dict[str, Any]]) -> httpx.Client:
>     """An httpx.Client that answers with the given responses in order and records each request body in calls."""
>
>     def handler(request: httpx.Request) -> httpx.Response:
>         calls.append(json.loads(request.content))
>         return responses[len(calls) - 1]
>
>     return httpx.Client(transport=httpx.MockTransport(handler))
>
>
> OK: dict[str, Any] = {
>     "model": "pplx-decider-v1.2",
>     "answers": {name: {"type": "noul", "noul": p} for name, p in CLEAN.items()},
>     "usage": {"input_tokens": 1234, "output_tokens": 6},
> }
>
>
> def test_ask_returns_probabilities_tokens_and_request_id_and_retries_a_429() -> None:
>     calls: list[dict[str, Any]] = []
>     responses = [
>         httpx.Response(429, json={"error": "slow down"}, headers={"retry-after": "0"}),
>         httpx.Response(200, json=OK, headers={"x-request-id": "rid-1"}),
>     ]
>     with fake_client(responses, calls) as client:
>         probs, tokens, rid = dc.ask(client, dc.single_body(b"png", "x"))
>     assert (probs, tokens, rid, len(calls)) == (CLEAN, 1234, "rid-1", 2)
>     assert calls[0] == calls[1]
>
>
> def test_ask_raises_on_a_400_and_on_a_200_missing_an_answer() -> None:
>     calls: list[dict[str, Any]] = []
>     with fake_client([httpx.Response(400, json={"error": {"message": "bad state"}})], calls) as client:
>         try:
>             dc.ask(client, dc.single_body(b"png", "x"))
>             raise AssertionError("expected DecisionsError")
>         except dc.DecisionsError as exc:
>             assert "400" in str(exc) and "bad state" in str(exc)
>     partial = {**OK, "answers": {k: v for k, v in OK["answers"].items() if k != "error_state"}}
>     with fake_client([httpx.Response(200, json=partial)], []) as client:
>         try:
>             dc.ask(client, dc.single_body(b"png", "x"))
>             raise AssertionError("expected DecisionsError")
>         except dc.DecisionsError as exc:
>             assert "cannot read" in str(exc)
>
>
> def test_max_height_keeps_every_width_inside_the_tile_budget() -> None:
>     for width in (390, 1280, 1440, 2048):
>         assert round(width / 32) * (vq.max_height(width) // 32) <= vq.MAX_TILES
>     assert vq.max_height(1280) == 1632
>     assert vq.slug("/pricing.html") == "pricing" and vq.slug("/") == "index"
>
>
> @pytest.mark.parametrize("invalid", [float("nan"), float("inf"), -0.1, 1.1])
> def test_invalid_probabilities_cannot_pass_the_gate(invalid: float) -> None:
>     body = {**OK, "answers": {**OK["answers"], "text_clipped": {"noul": invalid}}}
>     response = httpx.Response(200, text=json.dumps(body))
>     with fake_client([response], []) as client, pytest.raises(dc.DecisionsError, match="cannot read"):
>         dc.ask(client, dc.single_body(b"png", "x"))
>
>
> def test_oversized_page_is_not_silently_cropped() -> None:
>     from unittest.mock import MagicMock
>
>     browser = MagicMock()
>     page = browser.new_context.return_value.new_page.return_value
>     page.evaluate.return_value = 5000
>     with pytest.raises(ValueError, match="split it into sections"):
>         vq.screenshot(browser, "http://localhost/", 1280, 800)
>     page.screenshot.assert_not_called()
>     browser.new_context.return_value.close.assert_called_once()
>
>
> def test_screenshot_preserves_viewport_width_and_full_height() -> None:
>     from unittest.mock import MagicMock
>
>     browser = MagicMock()
>     page = browser.new_context.return_value.new_page.return_value
>     page.evaluate.return_value = 1200
>     vq.screenshot(browser, "http://localhost/", 390, 844)
>     page.screenshot.assert_called_once_with(full_page=True, clip={"x": 0, "y": 0, "width": 390, "height": 1200}, animations="disabled")
>
>
> def test_run_appends_report_to_job_summary(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
>     from unittest.mock import MagicMock
>
>     monkeypatch.setattr(vq, "sync_playwright", MagicMock())
>     monkeypatch.setattr(vq, "screenshot", lambda *args: b"png")
>     monkeypatch.setattr(dc, "ask", lambda *args: (dict(CLEAN), 123, "rid"))
>     summary = tmp_path / "summary.md"
>     summary.write_text("Existing summary\n")
>     monkeypatch.setenv("GITHUB_STEP_SUMMARY", str(summary))
>     args = vq.parse_args(["--url", "https://example.com", "--pages", "/", "--out", str(tmp_path / "out")])
>     assert vq.run(args, "unused-test-key") == 0
>     report = (tmp_path / "out" / "report.md").read_text()
>     assert summary.read_text() == "Existing summary\n" + report
>
>
> def test_site_server_serves_the_variant_as_theme_css() -> None:
>     server, url = vq.serve("pr")
>     try:
>         theme = urllib.request.urlopen(f"{url}/theme.css", timeout=5).read().decode()
>         page = urllib.request.urlopen(f"{url}/pricing.html", timeout=5).read().decode()
>     finally:
>         server.shutdown()
>     assert theme == (vq.SITE_DIR / "pr.css").read_text()
>     assert '<link rel="stylesheet" href="/theme.css">' in page
> ```

> **visual-qa.yml**
>
> ```yaml visual-qa.yml theme={null}
> # Save as .github/workflows/visual-qa.yml. The step summary on the job page shows the report
> # that visual_qa.py writes to $GITHUB_STEP_SUMMARY; the screenshots are uploaded as an artifact.
> name: Visual QA
> on:
>   pull_request:
>
> permissions:
>   contents: read
>
> jobs:
>   visual-qa:
>     if: github.event.pull_request.head.repo.full_name == github.repository
>     runs-on: ubuntu-latest
>     steps:
>       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
>       - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
>         with:
>           python-version: "3.12"
>       - name: Install
>         run: |
>           python -m pip install -r requirements.txt
>           python -m playwright install --with-deps chromium
>       - name: Visual QA gate
>         env:
>           PERPLEXITY_API_KEY: ${{ secrets.PERPLEXITY_API_KEY }}
>         # The bundled site: the pull request's stylesheet against main's. For your own site, build or deploy
>         # a preview first and use --url "$PREVIEW_URL" --baseline-url "$PRODUCTION_URL" instead.
>         run: python visual_qa.py --variant pr --baseline main
>       - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
>         if: always()
>         with:
>           name: visual-qa
>           path: out/
> ```