> ## Documentation Index
> Fetch the complete documentation index at: https://docs.perplexity.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Check a Chart for Misleading Design with a Decision Model
> Send a chart image to the Decisions API with yes/no questions about misleading design and your brand guide, and get a probability for each. Your code gates the checks by chart type, compares them to cutoffs, labels the chart, and prints why.
A line chart with two value axes can make marketing spend look like it drives signups. Each axis is scaled on its own, so the two lines land on top of each other no matter what the data says. A bar chart whose axis starts at 93 makes a 5% difference look like a 5x difference. No number on either chart is wrong. The drawing is, which is why nothing in your data pipeline catches it: the query is right, the totals reconcile, and the chart still tells the wrong story to the people deciding next quarter's budget.
This recipe sends a chart image to `pplx-decider-latest` with yes/no questions about how the chart is drawn: does the bar axis start above zero, do two value axes make unrelated series look linked, do pie labels add to more than 100%, and six more. The same request asks three questions about a short brand guide: are the data colors on palette, is there a source line, is the title left-aligned. The decision model returns a probability for each question, plus a chart type and an overall score. Your Python decides which checks apply, compares the probabilities to cutoffs, labels the chart, and prints the reason for every finding:
```text theme={null}
charts/spend_signups__misleading.png
design: misleading (line_or_area 0.99, overall 1.45 of 3)
0.99 Two value axes with mismatched scales
Why: Each axis was scaled separately, so the two lines track each
other by construction. The overlap suggests a link the data does not
show.
brand: conforms
```
A live run of the finished script on four of the test charts. The left pane shows the chart being checked. The terminal first prints the twelve checks the script sends with every image, then runs the checker and prints every probability the decision model returned. The honest temperature chart at the end starts its axis at 60 and still scores clear, because the script does not apply the truncated-baseline check to line charts.
The test pair behind that output. Same twelve months of spend and signups, same title, same source line. On the right, each series gets its own axis and the lines track each other. On the left, both are indexed to January on one axis, and spend grew 27% while signups grew 17%. In the recorded run, dual\_axis\_mismatch came back at 0.00 for the left chart and 0.99 for the right one.
You can point the finished script at any PNG, JPEG, or WebP: a screenshot, an export from Tableau or Looker, or a figure from a notebook. The page also shows how to measure a checker like this one. A script draws every test chart twice from the same data, once honest and once with a single planted problem, and writes the answer key before any API call. You tune on a development set, freeze the checklist and cutoffs, and score a held-out set once. The held-out run below caught 3 of 3 design problems and 3 of 3 brand violations with no false flags, and the page reports the one experimental check that missed, and why.
## Why a decision model
A chat model can look at the same chart and write a paragraph about it. You cannot put a cutoff on a paragraph, count how often it is right, or rerun it tomorrow and get the same answer. A decision model returns a calibrated probability for each question you ask, so your code can hold the policy: which checks apply to a bar chart, where `FLAG` sits, and what happens between `clear` and `misleading`. The policy is readable, testable, and yours to change without touching the model.
Three things follow from that, and this recipe uses all three:
* **Numbers you can evaluate.** Draw 41 charts with known problems, run them, and the result is a table: caught, passed, false flags. You know how good the checker is before you trust it.
* **One request per chart.** Fourteen questions about one image go in one request, come back in about half a second, and cost about four thousandths of a cent in input tokens. Output tokens are free. That is a nightly sweep of every chart your company publishes, not a sample.
* **Reasons in your words.** The model returns probabilities. Your checklist carries the sentence that explains each check, so the output says why a chart misleads in words you approved, not words a model generated.
## What you need
* Python 3.11 or newer on macOS or Linux. The shell commands use Unix syntax. On a Mac, check with `python3 --version`. The `python3` that ships with Apple's command line tools can be older; install a newer one with `brew install python@3.12` or from [python.org](https://www.python.org/downloads/macos/).
* A Perplexity API key. Every key can call the Decisions API; there is no separate signup.
The Decisions API is one JSON endpoint, `POST https://api.perplexity.ai/v1/decisions`, so the script calls it with `httpx`, a plain HTTP client. Pillow resizes images. matplotlib draws the test charts in the second half of the page. Nothing else is needed.
## Set up
Everything lives in one directory. Checking a chart takes three files: `requirements.txt`, `checklist.py`, and `chart_check.py`. Measuring the checker takes three more, which come later on the page.
```text theme={null}
decisions-api-chart-check/
├── requirements.txt # four pinned dependencies
├── checklist.py # the questions, the reasons, and the brand guide: edit this one
├── chart_check.py # the request, the policy, and the CLI
├── make_charts.py # draws the test charts and the answer key, no API calls
├── evaluate.py # scores chart_check.py against the answer key
├── test_chart_check.py # offline tests, no network
├── charts/ # written by make_charts.py
├── results.jsonl # appended by chart_check.py and evaluate.py
└── .venv/ # virtual environment from the install step
```
Start with the dependencies, pinned to the versions this recipe was tested with.
> **requirements.txt**
>
> ```text requirements.txt theme={null}
> httpx==0.28.1
> matplotlib==3.11.2
> pillow==13.0.0
> pytest==9.1.1
> ```
Create the directory and save `requirements.txt` in it. Then create a virtual environment, install the packages, and export your key.
> **Install and set the key**
>
> ```bash theme={null}
> mkdir decisions-api-chart-check && cd decisions-api-chart-check
> # save requirements.txt here, then continue
> python3 -m venv .venv
> source .venv/bin/activate
> python -m pip install -r requirements.txt
> export PERPLEXITY_API_KEY="your-api-key-here"
> ```
## The checklist
`checklist.py` is plain data, and it is the file you edit to make this checker yours.
`CHART_TYPES` becomes a `choice` question, so the response has one chart type and a confidence. Each entry in `CHECKS` becomes a `noul` (yes/no) question with written criteria for yes and for no, and the response has one probability per check. Each entry also carries a reason: one sentence, written by you, about what the problem does to a reader. The model never writes it; your code prints it next to the probability, so the output says why a chart misleads, in words you approved. `APPLIES_TO` lists the chart types each check makes sense for. `OVERALL_RUBRIC` becomes a `score` question whose four levels run from faithful to deceptive. `EXPERIMENTAL` names the checks that are measured but not trusted on their own; there is one, and the results section shows why.
`BRAND_GUIDE` is a short brand guide in plain text. It goes into the request next to the image, and each entry in `BRAND_CHECKS` asks whether the chart breaks one of its rules. A brand entry carries a fix instead of a reason. Replace the three rules with your own, but keep each one to something visible in the image: colors, a source line, title placement, a logo.
Write criteria the way you would brief a new analyst. "The first tick is not 0" and "for example 1990, 2000, 2010, 2015" give the model something concrete to look for. Cover the cases where the rule does not apply, too: the title rule says a chart with no title is fine, because without that sentence the model split the difference at about 0.6 on untitled charts. For every question, yes is the problem, so a high probability always means something to fix.
> **checklist.py**
>
> ```python checklist.py theme={null}
> """What chart_check.py asks about every chart. Edit this file to change the checks.
>
> Each design check is (short label, why it misleads, what yes means, what no means).
> Each brand rule is (short label, what to fix, what yes means, what no means).
> For every question, yes is the problem.
> """
>
> # Checks that are measured but not reliable enough to count in a headline number.
> # parts_dont_sum asks the model to add numbers across labels, and it scores a
> # 130% pie at 0.67 and a 123% donut at 0.04. Keep it visible, do not trust it alone.
> EXPERIMENTAL = {"parts_dont_sum"}
>
> CHART_TYPES = {
> "bar": "Vertical or horizontal bars or columns",
> "line_or_area": "Lines or filled areas over an ordered axis",
> "pie_or_donut": "Slices of a circle or ring",
> "scatter_or_bubble": "Points positioned by two values",
> "map": "Geographic map",
> "table_or_text": "Mostly numbers or text, not a plotted shape",
> "dashboard": "Several different charts shown together",
> "other": None,
> }
>
> CHECKS: dict[str, tuple[str, str, str, str]] = {
> "truncated_baseline": (
> "Bar axis does not start at zero",
> "Readers compare bar lengths. With the axis cut above zero, a small difference between bars looks several times larger than it is.",
> "A bar or column chart's value axis starts above zero (the first tick is not 0), so bar lengths exaggerate the differences between bars.",
> "The bars start at zero, or the chart has no bars.",
> ),
> "dual_axis_mismatch": (
> "Two value axes with mismatched scales",
> "Each axis was scaled separately, so the two lines track each other by construction. The overlap suggests a link the data does not show.",
> "Two series are plotted against two different value axes (one on the left, one on the right) with scales chosen so the series appear to move together, suggesting a relationship.",
> "There is one value axis, or the two axes share a scale or an index.",
> ),
> "inverted_axis": (
> "Value axis runs the wrong way",
> "Readers expect up to mean more. Flipped, a rising count looks like a fall.",
> "A value axis runs opposite to convention: larger values appear lower on the vertical axis or further left on the horizontal axis, so increases look like decreases.",
> "Values increase upward and to the right as usual.",
> ),
> "uneven_intervals": (
> "Irregular time intervals drawn as equal",
> "Ten-year and one-year gaps get the same width, so recent change looks steeper than earlier change.",
> "The horizontal axis shows time points that are unequally spaced in time (for example 1990, 2000, 2010, 2015, 2018, 2019) but drawn at equal distances, as if the gaps were the same.",
> "Time points are evenly spaced, or they are positioned in proportion to the actual gaps, or the axis is not time.",
> ),
> "parts_dont_sum": (
> "Pie or donut parts do not add to 100%",
> "A pie promises that the parts make a whole. Labels that add to more or less than 100% mean the slices cannot be trusted.",
> "A pie or donut chart's labeled percentages add up to clearly more or less than 100%, or slice sizes do not match their labels.",
> "The labeled parts add to about 100% and slice sizes match them, or the chart is not a pie or donut.",
> ),
> "title_overclaims": (
> "Title claims more than the data shows",
> "Most readers take the headline and skip the axis. The title promises a change the plotted values do not show.",
> "The title or a caption asserts a trend, surge, collapse, record or large difference that the plotted values do not support (for example calling a change of about one percentage point a surge).",
> "The title describes the plotted values fairly, or makes no claim about trend or size.",
> ),
> "missing_axis_labels": (
> "Value axis has no readable scale",
> "Without numbers on the axis, a reader cannot tell whether the differences are 1% or 100%.",
> "A value axis has no tick labels, so a reader cannot tell the magnitude or units of what is plotted.",
> "Value axes have tick labels, or the chart type needs none (a labeled pie chart, for example).",
> ),
> "perspective_3d": (
> "3D perspective distorts sizes",
> "Perspective makes nearer bars or slices look bigger and farther ones smaller, so the eye misreads the values.",
> "The chart is drawn in 3D or tilted perspective, so slices or bars nearer the viewer look larger than their values and farther ones look smaller.",
> "The chart is flat, with no perspective or depth effects.",
> ),
> "cumulative_disguise": (
> "Cumulative total hides a per-period decline",
> "A running total can only rise. Monthly numbers that are falling show up as a curve that still climbs.",
> "The chart plots a cumulative or running total that can only rise, while the subject (sales, downloads, users) is normally reported per period, so a slowdown or decline is hidden in a flattening curve.",
> "Values are shown per period, or a cumulative view is the natural reading of the subject.",
> ),
> }
>
> # A design check only counts on the chart types listed for it.
> APPLIES_TO: dict[str, set[str]] = {
> "truncated_baseline": {"bar", "dashboard"},
> "dual_axis_mismatch": {"line_or_area", "bar", "dashboard"},
> "inverted_axis": {"line_or_area", "bar", "scatter_or_bubble", "dashboard"},
> "uneven_intervals": {"line_or_area", "bar", "scatter_or_bubble", "dashboard"},
> "parts_dont_sum": {"pie_or_donut", "dashboard"},
> "title_overclaims": set(CHART_TYPES),
> "missing_axis_labels": {"line_or_area", "bar", "scatter_or_bubble", "dashboard"},
> "perspective_3d": {"bar", "pie_or_donut", "line_or_area", "dashboard", "other"},
> "cumulative_disguise": {"line_or_area", "bar", "dashboard"},
> }
>
> OVERALL_RUBRIC = [
> "Faithful: the design does not distort what the data shows",
> "Minor: a presentation flaw that could confuse but is unlikely to mislead",
> "Misleading: the design makes differences or trends look much larger or smaller than the data supports",
> "Deceptive: the design pushes a conclusion the data does not support",
> ]
>
> # Your brand guide, sent with every chart. Replace it with your own rules.
> BRAND_GUIDE = (
> "Chart standards: "
> "1. Data colors: bars, lines, areas, and slices use only dark teal, light teal, orange, and rust. "
> "No blue, green, red, purple, or yellow. Text, axes, and backgrounds can be black, white, or gray. "
> "2. Source: the small note at the bottom left of every chart names the data source after the word 'Source:'. "
> "3. Title: the title is bold and left-aligned above the chart, not centered."
> )
>
> BRAND_CHECKS: dict[str, tuple[str, str, str, str]] = {
> "off_brand_colors": (
> "Data colors are outside the brand palette",
> "Redraw the data in dark teal, light teal, orange, or rust.",
> "At least one bar, line, area, or slice is a color the brand guide does not allow, such as blue, green, red, purple, or yellow.",
> "Every bar, line, area, and slice is dark teal, light teal, orange, or rust.",
> ),
> "missing_source": (
> "No source line",
> "Add 'Source:' and the data source to the note at the bottom left.",
> "No text anywhere at the bottom of the chart contains the word 'Source:' followed by where the data came from.",
> "Somewhere in the small text at the bottom of the chart, the word 'Source:' is followed by where the data came from, even if other text comes first.",
> ),
> "centered_title": (
> "Title is not left-aligned",
> "Align the title to the left edge of the chart.",
> "The chart title is centered or right-aligned above the chart.",
> "The chart title is left-aligned above the chart, or the chart has no title.",
> ),
> }
> ```
## The script, stage by stage
The rest of this section walks through `chart_check.py` one piece at a time. Every block below is an exact excerpt of the full file, which follows the excerpts.
### Imports and constants
The endpoint, the model, and the four cutoffs live at the top. `pplx-decider-latest` selects the newest decision model. The cutoffs are the whole policy: a probability at or above `FLAG` means the problem is present, one between `BORDERLINE` and `FLAG` sends the chart to a person, and an overall score at or above `REVIEW_SCORE` does the same for design even when no single check fires. `TYPE_CONFIDENCE` sends a chart to review when the model is not sure what kind of chart it is, because the gating in `judge` depends on that answer.
> **Imports and constants**
>
> ```python theme={null}
> """Check chart images for misleading design and brand standards with a decision model.
>
> One Decisions API request per image returns a probability for each design
> check and each brand rule in checklist.py, plus a chart type and an overall
> score. The cutoffs and rules in this file turn those numbers into two labels:
> clear, review, or misleading for design, and conforms, review, or off-brand
> for the brand guide.
>
> python chart_check.py charts/revenue_bars__misleading.png
> python chart_check.py --no-brand deck/*.png # design checks only
> """
>
> from __future__ import annotations
>
> import argparse
> import base64
> import email.utils
> import io
> import json
> import os
> import sys
> import textwrap
> import time
> from dataclasses import asdict, dataclass, field
>
> import httpx
> from PIL import Image
>
> from checklist import APPLIES_TO, BRAND_CHECKS, BRAND_GUIDE, CHART_TYPES, CHECKS, EXPERIMENTAL, OVERALL_RUBRIC
>
> DECISIONS_URL = "https://api.perplexity.ai/v1/decisions"
> MODEL = "pplx-decider-latest"
>
> # Cutoffs. Freeze these before you score held-out charts.
> FLAG = 0.60 # at or above: the problem is present
> BORDERLINE = 0.40 # from here up to FLAG: a person should look
> REVIEW_SCORE = 1.5 # an overall score (0 to 3) this high alone sends a chart to review
> TYPE_CONFIDENCE = 0.5 # below this, the chart type is a guess and so is the gating: review
> ```
### Fitting the image budget
The API reads images in 32 × 32 pixel tiles and accepts up to 2,048 tiles per image. An image over the budget does not return `400`. The request waits about a minute and then returns `504` ([Decisions API docs](https://docs.perplexity.ai/docs/decisions/quickstart.md)). So resizing is a correctness step. A full-screen screenshot on a Retina display is often 2880 × 1800 pixels, over 5,000 tiles, and would time out without it.
`fit_to_tiles` rounds the width and height to the nearest multiple of 32, as the docs describe, and shrinks the image only when it is over budget. It flattens transparent images onto white first, because some exporters store black pixels under a transparent background, and a model would see a black chart. It also converts JPEG and WebP input to PNG, so the request always sends one format. The test charts are 900 × 560, about 500 tiles, and pass through unchanged.
> **fit\_to\_tiles()**
>
> ```python theme={null}
> def fit_to_tiles(image_bytes: bytes, max_tiles: int = 2048) -> bytes:
> """Return a PNG that fits the API's budget of 2,048 tiles of 32 x 32 pixels."""
> im = Image.open(io.BytesIO(image_bytes))
> if im.mode in ("RGBA", "LA", "P"): # flatten transparency onto white, not black
> rgba = im.convert("RGBA")
> im = Image.alpha_composite(Image.new("RGBA", rgba.size, "white"), rgba)
> im = im.convert("RGB")
> w, h = im.size
> tiles = max(1, round(w / 32)) * max(1, round(h / 32))
> if tiles > max_tiles:
> scale = (max_tiles / tiles) ** 0.5 * 0.98 # a little margin for rounding
> im = im.resize((max(32, int(w * scale)), max(32, int(h * scale))), Image.LANCZOS)
> out = io.BytesIO()
> im.save(out, format="PNG")
> return out.getvalue()
> ```
### One image, fourteen questions
`build_request` puts the image in `state` as a data URL, next to a one-line task and the brand guide. `state` accepts an object, so the image, the task, and the guide arrive as labeled fields. The API never fetches a URL; an `http` image URL returns `400`. Pass `context` to add a sentence such as the dashboard name. Pass `brand=False` to leave the guide and its three questions out, which is what the `--no-brand` flag does for charts that were never meant to follow your guide.
All fourteen questions go in one request: the chart type, nine design checks, three brand rules, and the overall score. They are facets of one review of one image, which is the case where the docs say grouping helps. Grouping also changes the numbers, because each question sees the others. Keep the checklist and its order fixed while you tune cutoffs; the results section has a real chart that moved from 0.61 to 0.45 when the three brand questions joined the request.
> **yes\_no() and build\_request()**
>
> ```python theme={null}
> def yes_no(yes: str, no: str) -> dict:
> return {"type": "noul", "instructions": "Judge only what is visible in the screenshot.", "criteria": {"true": yes, "false": no}}
>
>
> def build_request(png: bytes, context: str | None = None, brand: bool = True) -> dict:
> """One image, fourteen questions: chart type, nine design checks, three brand rules, overall."""
> url = "data:image/png;base64," + base64.b64encode(png).decode()
> state: dict = {
> "task": "Review this chart screenshot for misleading design choices" + (" and for the brand guide." if brand else "."),
> "chart": {"type": "image_url", "image_url": {"url": url}},
> }
> if brand:
> state["brand_guide"] = BRAND_GUIDE
> if context:
> state["context"] = context
> questions: dict = {"chart_type": {"type": "choice", "instructions": "What kind of chart is shown?", "criteria": CHART_TYPES}}
> for name, (_, _, yes, no) in CHECKS.items():
> questions[name] = yes_no(yes, no)
> for name, (_, _, yes, no) in BRAND_CHECKS.items() if brand else ():
> questions[name] = yes_no(yes, no)
> questions["overall"] = {
> "type": "score",
> "instructions": "How likely is a typical reader to draw a wrong conclusion from this chart because of how it is designed?",
> "criteria": OVERALL_RUBRIC,
> }
> return {"model": MODEL, "state": state, "questions": questions}
> ```
### Sending the request
`decide` posts the body with the client you pass in. On `429` it waits out `Retry-After`, which can be a number of seconds or an HTTP date; on a `5xx` it backs off and retries, up to three times. Anything else that is not `200` raises with the status, the start of the error body, and the `x-request-id` header, which is what support asks for. By default an organization can send 10 requests per second; this script sends one at a time.
> **retry\_after() and decide()**
>
> ```python theme={null}
> def retry_after(value: str | None) -> float:
> """Seconds to wait from a Retry-After header, which can be a number or an HTTP date."""
> if value is None:
> return 1.0
> try:
> return max(0.0, float(value))
> except ValueError:
> date = email.utils.parsedate_to_datetime(value)
> return max(0.0, date.timestamp() - time.time()) if date else 1.0
>
>
> def decide(client: httpx.Client, body: dict, retries: int = 3) -> tuple[dict, httpx.Response]:
> """POST one request. Wait out 429s and retry 5xx with backoff."""
> for attempt in range(retries + 1):
> response = client.post(DECISIONS_URL, json=body)
> if attempt < retries and response.status_code == 429:
> time.sleep(retry_after(response.headers.get("Retry-After")))
> elif attempt < retries and response.status_code >= 500:
> time.sleep(2**attempt)
> else:
> break
> if response.status_code != 200:
> request_id = response.headers.get("x-request-id", "none")
> raise RuntimeError(f"{response.status_code} {response.text[:300]} (x-request-id {request_id})")
> return response.json(), response
> ```
### From probabilities to two labels
The model answers every design check for every chart, including checks that make no sense for it. A line chart whose axis starts at 60 °F is fine; the same axis on a bar chart is not. `judge` counts a design check only when the chart type the model returned is listed for it in `APPLIES_TO`. A high probability on a check that does not apply goes to `suppressed`, where you can still see it. Brand rules apply to every chart.
Then `judge` sorts the findings by probability and picks two labels. For design, any flag makes the chart `misleading`, and a borderline check, a high overall score, or an uncertain chart type makes it `review`. For brand, any flag makes it `off-brand` and a borderline rule makes it `review`; with `--no-brand` the brand label is `skipped`. Everything else is `clear` and `conforms`. The two labels are independent: a chart can be on-brand and misleading, or honest and off-brand. `misleading` still means a person should decide; it means the pattern is there with high probability, not that someone meant to deceive.
> **Verdict, label(), and judge()**
>
> ```python theme={null}
> @dataclass
> class Verdict:
> label: str # design: clear | review | misleading
> brand: str # brand guide: conforms | review | off-brand, or skipped with --no-brand
> chart_type: str
> chart_confidence: float
> overall: float
> flagged: list[str] = field(default_factory=list) # design checks at or above FLAG that apply
> borderline: list[str] = field(default_factory=list) # design checks from BORDERLINE to FLAG that apply
> suppressed: list[str] = field(default_factory=list) # high, but not for this chart type
> brand_flagged: list[str] = field(default_factory=list) # brand rules at or above FLAG
> brand_borderline: list[str] = field(default_factory=list)
> probabilities: dict[str, float] = field(default_factory=dict)
>
>
> def label(flagged: list, borderline: list, bad: str, good: str, extra_review: bool = False) -> str:
> return bad if flagged else "review" if borderline or extra_review else good
>
>
> def judge(answers: dict) -> Verdict:
> """Application policy: gate design checks by chart type, then apply the cutoffs."""
> chart_type, confidence = answers["chart_type"]["choice"], float(answers["chart_type"]["confidence"])
> probs = {name: float(answers[name]["noul"]) for name in [*CHECKS, *BRAND_CHECKS] if name in answers}
> overall = float(answers["overall"]["score"])
> v = Verdict("", "", chart_type, confidence, overall, probabilities=probs)
> for name, p in sorted(probs.items(), key=lambda item: -item[1]):
> if name in BRAND_CHECKS:
> if p >= FLAG:
> v.brand_flagged.append(name)
> elif p >= BORDERLINE:
> v.brand_borderline.append(name)
> elif p >= FLAG:
> (v.flagged if chart_type in APPLIES_TO[name] else v.suppressed).append(name)
> elif p >= BORDERLINE and chart_type in APPLIES_TO[name]:
> v.borderline.append(name)
> # A guessed chart type means guessed gating, so a person looks even when nothing fired.
> v.label = label(v.flagged, v.borderline, "misleading", "clear", overall >= REVIEW_SCORE or confidence < TYPE_CONFIDENCE)
> v.brand = label(v.brand_flagged, v.brand_borderline, "off-brand", "conforms") if any(n in probs for n in BRAND_CHECKS) else "skipped"
> return v
> ```
### The report and the command line
`check_file` reads an image, fits it, sends it, and returns a flat record with both labels, every probability, the input tokens, the time, and the request id. `explain` turns each finding into two parts: the probability with the check's short label, then the reason or fix from `checklist.py`. `print_record` prints the design label with a `Why:` line per finding and the brand label with a `Fix:` line per broken rule.
`main` reads the key from `PERPLEXITY_API_KEY`, opens one `httpx.Client` that sends it as a bearer token, checks each file in order, and appends one JSON line per image to `results.jsonl`. A failure stops the run with a one-line message and writes nothing.
> **check\_file(), explain(), print\_record(), and main()**
>
> ```python theme={null}
> def check_file(client: httpx.Client, path: str, brand: bool = True) -> dict:
> """Read, resize, and send one image. Return a flat record for results.jsonl."""
> with open(path, "rb") as handle:
> body = build_request(fit_to_tiles(handle.read()), brand=brand)
> started = time.perf_counter()
> data, response = decide(client, body)
> verdict = judge(data["answers"])
> return {
> "file": path,
> **asdict(verdict),
> "input_tokens": data["usage"]["input_tokens"],
> "seconds": round(time.perf_counter() - started, 2),
> "request_id": response.headers.get("x-request-id"),
> "model": MODEL,
> "checked_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
> }
>
>
> def explain(names: list[str], table: dict, probs: dict, prefix: str) -> list[str]:
> """One line per finding: the probability, the label, and the reason or fix from checklist.py."""
> lines = []
> for name in names:
> short, reason = table[name][0], table[name][1]
> lines.append(f" {probs[name]:.2f} {short}" + (" (experimental check)" if name in EXPERIMENTAL else ""))
> lines += textwrap.wrap(f"{prefix}{reason}", 78, initial_indent=" ", subsequent_indent=" ")
> return lines
>
>
> def print_record(r: dict) -> None:
> p = r["probabilities"]
> lines = [r["file"], f" design: {r['label']} ({r['chart_type']} {r['chart_confidence']:.2f}, overall {r['overall']:.2f} of 3)"]
> lines += explain(r["flagged"] + r["borderline"], CHECKS, p, "Why: ")
> if r["brand"] != "skipped":
> lines += [f" brand: {r['brand']}"] + explain(r["brand_flagged"] + r["brand_borderline"], BRAND_CHECKS, p, "Fix: ")
> print("\n".join(lines))
> print(f" {r['input_tokens']:,} input tokens, {r['seconds']:.2f} s, request {r['request_id']}")
>
>
> def main(argv: list[str] | None = None) -> int:
> parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
> parser.add_argument("files", nargs="+", help="PNG, JPEG, or WebP chart images")
> parser.add_argument("--out", default="results.jsonl", help="append one JSON line per image")
> parser.add_argument("--no-brand", action="store_true", help="skip the brand guide; design checks only")
> args = parser.parse_args(argv)
>
> api_key = os.environ.get("PERPLEXITY_API_KEY")
> if not api_key:
> print("Set PERPLEXITY_API_KEY first.", file=sys.stderr)
> return 2
> records = []
> with httpx.Client(headers={"Authorization": f"Bearer {api_key}"}, timeout=90.0) as client:
> for path in args.files:
> try:
> records.append(check_file(client, path, brand=not args.no_brand))
> except (RuntimeError, httpx.HTTPError) as error:
> print(f"{path}: Decisions API failed: {error}", file=sys.stderr)
> return 1
> print_record(records[-1])
> with open(args.out, "a", encoding="utf-8") as handle:
> handle.writelines(json.dumps(r) + "\n" for r in records)
> return 0
>
>
> if __name__ == "__main__":
> sys.exit(main())
> ```
### The full file
Save this as `chart_check.py`. It is the excerpts above in order, nothing else.
> **Full code - chart\_check.py**
>
> ```python chart_check.py theme={null}
> """Check chart images for misleading design and brand standards with a decision model.
>
> One Decisions API request per image returns a probability for each design
> check and each brand rule in checklist.py, plus a chart type and an overall
> score. The cutoffs and rules in this file turn those numbers into two labels:
> clear, review, or misleading for design, and conforms, review, or off-brand
> for the brand guide.
>
> python chart_check.py charts/revenue_bars__misleading.png
> python chart_check.py --no-brand deck/*.png # design checks only
> """
>
> from __future__ import annotations
>
> import argparse
> import base64
> import email.utils
> import io
> import json
> import os
> import sys
> import textwrap
> import time
> from dataclasses import asdict, dataclass, field
>
> import httpx
> from PIL import Image
>
> from checklist import APPLIES_TO, BRAND_CHECKS, BRAND_GUIDE, CHART_TYPES, CHECKS, EXPERIMENTAL, OVERALL_RUBRIC
>
> DECISIONS_URL = "https://api.perplexity.ai/v1/decisions"
> MODEL = "pplx-decider-latest"
>
> # Cutoffs. Freeze these before you score held-out charts.
> FLAG = 0.60 # at or above: the problem is present
> BORDERLINE = 0.40 # from here up to FLAG: a person should look
> REVIEW_SCORE = 1.5 # an overall score (0 to 3) this high alone sends a chart to review
> TYPE_CONFIDENCE = 0.5 # below this, the chart type is a guess and so is the gating: review
>
>
> def fit_to_tiles(image_bytes: bytes, max_tiles: int = 2048) -> bytes:
> """Return a PNG that fits the API's budget of 2,048 tiles of 32 x 32 pixels."""
> im = Image.open(io.BytesIO(image_bytes))
> if im.mode in ("RGBA", "LA", "P"): # flatten transparency onto white, not black
> rgba = im.convert("RGBA")
> im = Image.alpha_composite(Image.new("RGBA", rgba.size, "white"), rgba)
> im = im.convert("RGB")
> w, h = im.size
> tiles = max(1, round(w / 32)) * max(1, round(h / 32))
> if tiles > max_tiles:
> scale = (max_tiles / tiles) ** 0.5 * 0.98 # a little margin for rounding
> im = im.resize((max(32, int(w * scale)), max(32, int(h * scale))), Image.LANCZOS)
> out = io.BytesIO()
> im.save(out, format="PNG")
> return out.getvalue()
>
>
> def yes_no(yes: str, no: str) -> dict:
> return {"type": "noul", "instructions": "Judge only what is visible in the screenshot.", "criteria": {"true": yes, "false": no}}
>
>
> def build_request(png: bytes, context: str | None = None, brand: bool = True) -> dict:
> """One image, fourteen questions: chart type, nine design checks, three brand rules, overall."""
> url = "data:image/png;base64," + base64.b64encode(png).decode()
> state: dict = {
> "task": "Review this chart screenshot for misleading design choices" + (" and for the brand guide." if brand else "."),
> "chart": {"type": "image_url", "image_url": {"url": url}},
> }
> if brand:
> state["brand_guide"] = BRAND_GUIDE
> if context:
> state["context"] = context
> questions: dict = {"chart_type": {"type": "choice", "instructions": "What kind of chart is shown?", "criteria": CHART_TYPES}}
> for name, (_, _, yes, no) in CHECKS.items():
> questions[name] = yes_no(yes, no)
> for name, (_, _, yes, no) in BRAND_CHECKS.items() if brand else ():
> questions[name] = yes_no(yes, no)
> questions["overall"] = {
> "type": "score",
> "instructions": "How likely is a typical reader to draw a wrong conclusion from this chart because of how it is designed?",
> "criteria": OVERALL_RUBRIC,
> }
> return {"model": MODEL, "state": state, "questions": questions}
>
>
> def retry_after(value: str | None) -> float:
> """Seconds to wait from a Retry-After header, which can be a number or an HTTP date."""
> if value is None:
> return 1.0
> try:
> return max(0.0, float(value))
> except ValueError:
> date = email.utils.parsedate_to_datetime(value)
> return max(0.0, date.timestamp() - time.time()) if date else 1.0
>
>
> def decide(client: httpx.Client, body: dict, retries: int = 3) -> tuple[dict, httpx.Response]:
> """POST one request. Wait out 429s and retry 5xx with backoff."""
> for attempt in range(retries + 1):
> response = client.post(DECISIONS_URL, json=body)
> if attempt < retries and response.status_code == 429:
> time.sleep(retry_after(response.headers.get("Retry-After")))
> elif attempt < retries and response.status_code >= 500:
> time.sleep(2**attempt)
> else:
> break
> if response.status_code != 200:
> request_id = response.headers.get("x-request-id", "none")
> raise RuntimeError(f"{response.status_code} {response.text[:300]} (x-request-id {request_id})")
> return response.json(), response
>
>
> @dataclass
> class Verdict:
> label: str # design: clear | review | misleading
> brand: str # brand guide: conforms | review | off-brand, or skipped with --no-brand
> chart_type: str
> chart_confidence: float
> overall: float
> flagged: list[str] = field(default_factory=list) # design checks at or above FLAG that apply
> borderline: list[str] = field(default_factory=list) # design checks from BORDERLINE to FLAG that apply
> suppressed: list[str] = field(default_factory=list) # high, but not for this chart type
> brand_flagged: list[str] = field(default_factory=list) # brand rules at or above FLAG
> brand_borderline: list[str] = field(default_factory=list)
> probabilities: dict[str, float] = field(default_factory=dict)
>
>
> def label(flagged: list, borderline: list, bad: str, good: str, extra_review: bool = False) -> str:
> return bad if flagged else "review" if borderline or extra_review else good
>
>
> def judge(answers: dict) -> Verdict:
> """Application policy: gate design checks by chart type, then apply the cutoffs."""
> chart_type, confidence = answers["chart_type"]["choice"], float(answers["chart_type"]["confidence"])
> probs = {name: float(answers[name]["noul"]) for name in [*CHECKS, *BRAND_CHECKS] if name in answers}
> overall = float(answers["overall"]["score"])
> v = Verdict("", "", chart_type, confidence, overall, probabilities=probs)
> for name, p in sorted(probs.items(), key=lambda item: -item[1]):
> if name in BRAND_CHECKS:
> if p >= FLAG:
> v.brand_flagged.append(name)
> elif p >= BORDERLINE:
> v.brand_borderline.append(name)
> elif p >= FLAG:
> (v.flagged if chart_type in APPLIES_TO[name] else v.suppressed).append(name)
> elif p >= BORDERLINE and chart_type in APPLIES_TO[name]:
> v.borderline.append(name)
> # A guessed chart type means guessed gating, so a person looks even when nothing fired.
> v.label = label(v.flagged, v.borderline, "misleading", "clear", overall >= REVIEW_SCORE or confidence < TYPE_CONFIDENCE)
> v.brand = label(v.brand_flagged, v.brand_borderline, "off-brand", "conforms") if any(n in probs for n in BRAND_CHECKS) else "skipped"
> return v
>
>
> def check_file(client: httpx.Client, path: str, brand: bool = True) -> dict:
> """Read, resize, and send one image. Return a flat record for results.jsonl."""
> with open(path, "rb") as handle:
> body = build_request(fit_to_tiles(handle.read()), brand=brand)
> started = time.perf_counter()
> data, response = decide(client, body)
> verdict = judge(data["answers"])
> return {
> "file": path,
> **asdict(verdict),
> "input_tokens": data["usage"]["input_tokens"],
> "seconds": round(time.perf_counter() - started, 2),
> "request_id": response.headers.get("x-request-id"),
> "model": MODEL,
> "checked_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
> }
>
>
> def explain(names: list[str], table: dict, probs: dict, prefix: str) -> list[str]:
> """One line per finding: the probability, the label, and the reason or fix from checklist.py."""
> lines = []
> for name in names:
> short, reason = table[name][0], table[name][1]
> lines.append(f" {probs[name]:.2f} {short}" + (" (experimental check)" if name in EXPERIMENTAL else ""))
> lines += textwrap.wrap(f"{prefix}{reason}", 78, initial_indent=" ", subsequent_indent=" ")
> return lines
>
>
> def print_record(r: dict) -> None:
> p = r["probabilities"]
> lines = [r["file"], f" design: {r['label']} ({r['chart_type']} {r['chart_confidence']:.2f}, overall {r['overall']:.2f} of 3)"]
> lines += explain(r["flagged"] + r["borderline"], CHECKS, p, "Why: ")
> if r["brand"] != "skipped":
> lines += [f" brand: {r['brand']}"] + explain(r["brand_flagged"] + r["brand_borderline"], BRAND_CHECKS, p, "Fix: ")
> print("\n".join(lines))
> print(f" {r['input_tokens']:,} input tokens, {r['seconds']:.2f} s, request {r['request_id']}")
>
>
> def main(argv: list[str] | None = None) -> int:
> parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
> parser.add_argument("files", nargs="+", help="PNG, JPEG, or WebP chart images")
> parser.add_argument("--out", default="results.jsonl", help="append one JSON line per image")
> parser.add_argument("--no-brand", action="store_true", help="skip the brand guide; design checks only")
> args = parser.parse_args(argv)
>
> api_key = os.environ.get("PERPLEXITY_API_KEY")
> if not api_key:
> print("Set PERPLEXITY_API_KEY first.", file=sys.stderr)
> return 2
> records = []
> with httpx.Client(headers={"Authorization": f"Bearer {api_key}"}, timeout=90.0) as client:
> for path in args.files:
> try:
> records.append(check_file(client, path, brand=not args.no_brand))
> except (RuntimeError, httpx.HTTPError) as error:
> print(f"{path}: Decisions API failed: {error}", file=sys.stderr)
> return 1
> print_record(records[-1])
> with open(args.out, "a", encoding="utf-8") as handle:
> handle.writelines(json.dumps(r) + "\n" for r in records)
> return 0
>
>
> if __name__ == "__main__":
> sys.exit(main())
> ```
## Check your own chart
With the virtual environment active and the key exported, point the script at any chart image. Use `--no-brand` for a chart that was never meant to follow your brand guide, and pass several files to check them in one run.
> **Check a chart**
>
> ```bash theme={null}
> python chart_check.py --no-brand ~/Desktop/chart.png
> ```
To see it on a chart you did not draw, save the [truncated bar graph from Wikimedia Commons](https://commons.wikimedia.org/wiki/File:Truncated_Bar_Graph.svg) as a PNG (the file page has a PNG preview at several widths) and check it. Output observed on 2026-10-19 against `pplx-decider-latest`. The image checked here was a 1783 × 1114 pixel screenshot of the preview; a smaller download costs fewer tokens and can move the probabilities a little:
> **Observed output**
>
> ```text theme={null}
> truncated_bars.png
> design: misleading (bar 0.98, overall 1.55 of 3)
> 0.98 Bar axis does not start at zero
> Why: Readers compare bar lengths. With the axis cut above zero, a
> small difference between bars looks several times larger than it is.
> 2,571 input tokens, 1.00 s, request 5945f866-cf90-4ae1-ae8c-02eccaf82e70
> ```
The script printed that from this response. The image is a bar chart, the bar axis is cut, and the other eight design checks came back near zero. `overall` is a `score` question, so it returns a position on the 0 to 3 rubric with a probability for each level.
> **The response behind it, abbreviated**
>
> ```json theme={null}
> {
> "model": "pplx-decider-latest",
> "answers": {
> "chart_type": {
> "type": "choice",
> "choice": "bar",
> "confidence": 0.983,
> "probabilities": {"bar": 0.985, "line_or_area": 0.009, "pie_or_donut": 0.003, ...}
> },
> "truncated_baseline": {"type": "noul", "noul": 0.983},
> "dual_axis_mismatch": {"type": "noul", "noul": 0.001},
> "inverted_axis": {"type": "noul", "noul": 0.000},
> ... six more design checks, each under 0.002 ...
> "overall": {
> "type": "score",
> "score": 1.546,
> "confidence": 0.456,
> "legend": {"0": "Faithful: ...", "1": "Minor: ...", "2": "Misleading: ...", "3": "Deceptive: ..."},
> "probabilities": {"0": 0.147, "1": 0.204, "2": 0.603, "3": 0.045}
> }
> },
> "usage": {"input_tokens": 2571, "output_tokens": 11}
> }
> ```
Everything from the first line of output to the last is a decision your code made from those numbers: that `truncated_baseline` applies to a bar chart, that 0.98 is over `FLAG`, that the label is `misleading`, and which sentence to print under it.
## Measure it
The rest of the page is about knowing how good the checker is. That takes charts with known problems.
### The test charts
`make_charts.py` draws 41 PNG files from 22 cases and writes `charts/labels.json`, the answer key. Each case is an honest chart plus the few settings its planted version changes, so you can read exactly what was planted: `revenue_bars` changes only `ylim`, `csat_title` changes only `title`, `plan_colors` changes only `colors`. The data, the chart kind, the labels, and the source line stay the same, so the only way to tell the two versions apart is the drawing.
Every chart follows the brand guide unless its case plants a brand violation, so the misleading charts are on-brand and the off-brand charts are honest. That keeps the two kinds of check from leaning on each other.
The cases fall into three sets:
* **Development (12 pairs):** one pair for each of the nine design checks and each of the three brand rules. Tune on these.
* **Controls (3 honest charts):** a temperature line chart with a 60 to 100 °F axis, a labeled log-scale chart, and a chart with no title. All three are fine designs that a careless checker flags.
* **Held out (7 pairs):** dark theme, horizontal bars, a donut, an area chart, a pie. Score these once, after everything is frozen.
The answer key is written before any chart is drawn, and no file name, title, or note in an image says which version it is. The API sees pixels only. Run it once; it makes no network calls.
> **make\_charts.py**
>
> ```python make_charts.py theme={null}
> """Draw the test charts: an honest version and a planted-problem version of the same data.
>
> Design cases write charts/__honest.png and charts/__misleading.png.
> Brand cases write charts/__honest.png and charts/__off_brand.png.
> charts/labels.json records the planted problem for every image. Nothing here
> calls the API, and nothing in an image says which version it is.
> """
>
> from __future__ import annotations
>
> import json
> from pathlib import Path
>
> import matplotlib
>
> matplotlib.use("Agg") # draw to files, no window
> import matplotlib.pyplot as plt
>
> OUT = Path("charts")
> BRAND = ["#20808D", "#8FD1D8", "#E6772E", "#A84B2F"] # dark teal, light teal, orange, rust
> OFF_BRAND = ["#1f77b4", "#d62728", "#2ca02c", "#9467bd"] # matplotlib defaults: blue, red, green, purple
> MONTHS = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]
> YEARS = ["1990", "2000", "2010", "2015", "2018", "2019", "2020"]
> SPEND = [40, 42, 41, 45, 44, 47, 46, 48, 50, 49, 52, 51]
> SIGNUPS = [1200, 1180, 1250, 1230, 1300, 1290, 1350, 1340, 1380, 1400, 1420, 1410]
> INCIDENTS = [120, 135, 150, 170, 185, 210, 235, 260, 290, 310]
> MONTHLY = [50, 46, 41, 37, 33, 29, 26, 23, 20]
> CUMULATIVE = [sum(MONTHLY[: i + 1]) for i in range(len(MONTHLY))]
> WEEKS = [f"W{i}" for i in range(1, 9)]
> UNITS = [310, 340, 365, 390, 420, 410, 450, 470]
>
>
> def index(xs: list[float]) -> list[float]:
> """Rescale a series so its first value is 100."""
> return [round(v / xs[0] * 100, 1) for v in xs]
>
>
> # Each case is the honest chart plus the few keys the misleading version changes.
> # Controls have no misleading version. "set" decides when a case is scored.
> CASES = [
> {"id": "revenue_bars", "set": "dev", "pattern": "truncated_baseline",
> "honest": {"kind": "bar", "title": "Revenue by quarter, FY2026", "note": "Net revenue, $ millions. Source: Finance data mart.",
> "x": ["Q1", "Q2", "Q3", "Q4"], "y": [94.1, 95.8, 96.9, 98.7], "ylabel": "$ millions"},
> "misleading": {"ylim": (93, 99.5)}},
> {"id": "spend_signups", "set": "dev", "pattern": "dual_axis_mismatch",
> "honest": {"kind": "line", "title": "Marketing spend and new signups, 2026", "note": "Monthly. Source: Growth dashboard.",
> "x": MONTHS, "y": index(SPEND), "label": "Marketing spend (index)",
> "y_extra": index(SIGNUPS), "label_extra": "New signups (index)", "ylabel": "Index (Jan = 100)", "ylim": (90, 135)},
> "misleading": {"y": SPEND, "label": "Marketing spend ($K)", "ylabel": "Marketing spend ($K)", "ylim": (38, 54),
> "y_extra": None, "y2": SIGNUPS, "label2": "New signups", "ylim2": (1150, 1450)}},
> {"id": "incidents_inverted", "set": "dev", "pattern": "inverted_axis",
> "honest": {"kind": "area", "title": "Security incidents per year", "note": "Confirmed incidents, 2015 to 2024. Source: Security operations.",
> "x": [str(y) for y in range(2015, 2025)], "y": INCIDENTS, "ylabel": "Incidents"},
> "misleading": {"invert": True}},
> {"id": "uneven_years", "set": "dev", "pattern": "uneven_intervals",
> "honest": {"kind": "line", "title": "Average home price", "note": "Metro area, $ thousands. Source: County assessor records.",
> "x": [int(y) for y in YEARS], "y": [120, 165, 240, 280, 330, 345, 390], "ylabel": "$ thousands", "xlabel": "Year"},
> "misleading": {"x": YEARS}}, # strings: matplotlib spaces categories evenly
> {"id": "market_share_pie", "set": "dev", "pattern": "parts_dont_sum",
> "honest": {"kind": "pie", "title": "Market share, 2026", "note": "Share of units sold. Source: Industry panel data.",
> "x": ["Our brand", "Competitor A", "Competitor B"], "y": [45, 35, 20]},
> "misleading": {"y": [52, 41, 37]}}, # labels add to 130%
> {"id": "csat_title", "set": "dev", "pattern": "title_overclaims",
> "honest": {"kind": "line", "title": "Customer satisfaction steady at 81 to 82%", "note": "CSAT score, monthly, n = 4,200 per month. Source: Support survey.",
> "x": MONTHS[:6], "y": [81.2, 81.5, 81.1, 81.9, 81.7, 82.0], "ylabel": "CSAT (%)", "ylim": (0, 100)},
> "misleading": {"title": "Customer satisfaction surges to a record high"}},
> {"id": "unlabeled_axis", "set": "dev", "pattern": "missing_axis_labels",
> "honest": {"kind": "bar", "title": "Churn by plan", "note": "Trailing 90 days. Source: Billing system.",
> "x": ["Free", "Starter", "Team", "Enterprise"], "y": [8.4, 5.1, 3.2, 1.1], "ylabel": "Churn (%)"},
> "misleading": {"hide_ticks": True, "ylabel": ""}},
> {"id": "cumulative_downloads", "set": "dev", "pattern": "cumulative_disguise",
> "honest": {"kind": "area", "title": "App downloads, 2026", "note": "Monthly. Source: App store console.",
> "x": MONTHS[:9], "y": MONTHLY, "ylabel": "Downloads per month (thousands)"},
> "misleading": {"y": CUMULATIVE, "ylabel": "Cumulative downloads (thousands)"}},
> {"id": "regional_3d", "set": "dev", "pattern": "perspective_3d",
> "honest": {"kind": "bar", "title": "Sales by region, Q3 2026", "note": "Share of unit sales, percent. Source: Sales ops.",
> "x": ["North", "South", "East", "West"], "y": [31, 27, 24, 18], "ylabel": "Share of sales (%)"},
> "misleading": {"kind": "bar3d"}},
> {"id": "plan_colors", "set": "dev", "brand": "off_brand_colors",
> "honest": {"kind": "bar", "title": "Active seats by plan", "note": "October 2026, thousands. Source: Billing system.",
> "x": ["Free", "Starter", "Team", "Enterprise"], "y": [42, 18, 11, 6], "ylabel": "Seats (thousands)"},
> "off_brand": {"colors": OFF_BRAND}},
> {"id": "latency_source", "set": "dev", "brand": "missing_source",
> "honest": {"kind": "line", "title": "API p95 latency by week", "note": "Milliseconds. Source: Gateway metrics.",
> "x": WEEKS, "y": [212, 205, 198, 201, 190, 186, 181, 179], "ylabel": "p95 latency (ms)"},
> "off_brand": {"note": ""}},
> {"id": "nps_title", "set": "dev", "brand": "centered_title",
> "honest": {"kind": "bar", "title": "Net Promoter Score by region", "note": "Q3 2026. Source: Customer survey.",
> "x": ["Americas", "EMEA", "APAC"], "y": [41, 37, 44], "ylabel": "NPS"},
> "off_brand": {"center_title": True}},
> {"id": "temp_line_nonzero", "set": "control", "pattern": None,
> "honest": {"kind": "line", "title": "Average daily temperature, Phoenix, first week of October", "note": "Degrees Fahrenheit. Source: NOAA station PHX.",
> "x": [f"Oct {d}" for d in range(1, 8)], "y": [88, 87, 85, 86, 84, 83, 85], "ylabel": "°F", "ylim": (60, 100)}},
> {"id": "no_title", "set": "control", "pattern": None,
> "honest": {"kind": "bar", "title": "", "note": "Open tickets by priority, today. Source: Help desk.",
> "x": ["P0", "P1", "P2", "P3"], "y": [3, 18, 44, 61], "ylabel": "Open tickets"}},
> {"id": "log_scale_labeled", "set": "control", "pattern": None,
> "honest": {"kind": "line", "title": "Requests per day, 2021 to 2026 (log scale)", "note": "Daily API requests, logarithmic axis. Source: Gateway metrics.",
> "x": [str(y) for y in range(2021, 2027)], "y": [12e3, 48e3, 210e3, 1.1e6, 5.2e6, 24e6],
> "ylabel": "Requests per day (log scale)", "log": True}},
> {"id": "h_margin_bars", "set": "holdout", "pattern": "truncated_baseline", "dark": True,
> "honest": {"kind": "hbar", "title": "Gross margin by product line", "note": "FY2026, percent. Source: Finance.",
> "x": ["Hardware", "Software", "Services", "Support"], "y": [62, 71, 66, 74], "ylabel": "Gross margin (%)", "ylim": (0, 100)},
> "misleading": {"ylim": (60, 75)}},
> {"id": "h_dual_climate", "set": "holdout", "pattern": "dual_axis_mismatch", "dark": True,
> "honest": {"kind": "line", "title": "Store temperature and ice cream units sold", "x": WEEKS, "y": UNITS, "label": "Units sold", "ylabel": "Units sold",
> "note": "Weekly, summer 2026. Store temperature ranged 72 to 79 °F over the same weeks. Source: Store sensors, POS."},
> "misleading": {"y": [72, 73, 74, 75, 77, 76, 78, 79], "label": "Store temperature (°F)", "ylabel": "Temperature (°F)",
> "ylim": (71, 80), "y2": UNITS, "label2": "Units sold", "ylim2": (300, 480), "note": "Weekly, summer 2026. Source: Store sensors, POS."}},
> {"id": "h_pie_sum", "set": "holdout", "pattern": "parts_dont_sum", "dark": True,
> "honest": {"kind": "donut", "title": "How teams spend their week", "note": "Share of reported hours. Source: Internal time survey.",
> "x": ["Meetings", "Focus work", "Email and chat", "Other"], "y": [28, 42, 18, 12]},
> "misleading": {"y": [38, 42, 28, 15]}}, # labels add to 123%
> {"id": "h_title_claim", "set": "holdout", "pattern": "title_overclaims", "dark": True,
> "honest": {"kind": "bar", "title": "Reported thefts per 1,000 residents, roughly unchanged", "note": "By year. Source: City police annual report.",
> "x": ["2022", "2023", "2024", "2025"], "y": [4.1, 4.0, 4.2, 4.3], "ylabel": "Per 1,000 residents", "ylim": (0, 6)},
> "misleading": {"title": "Theft rate explodes across the city"}},
> {"id": "h_signup_colors", "set": "holdout", "brand": "off_brand_colors",
> "honest": {"kind": "area", "title": "Weekly signups", "note": "Thousands. Source: Growth dashboard.",
> "x": WEEKS, "y": [8.2, 8.9, 9.4, 9.1, 10.3, 10.8, 11.6, 12.0], "ylabel": "Signups (thousands)"},
> "off_brand": {"colors": ["#9467bd"]}},
> {"id": "h_share_title", "set": "holdout", "brand": "centered_title", "dark": True,
> "honest": {"kind": "pie", "title": "Support tickets by channel", "note": "September 2026. Source: Help desk.",
> "x": ["Email", "Chat", "Phone"], "y": [48, 37, 15]},
> "off_brand": {"center_title": True}},
> {"id": "h_cost_source", "set": "holdout", "brand": "missing_source", "dark": True,
> "honest": {"kind": "hbar", "title": "Cloud spend by team", "note": "September 2026, $ thousands. Source: Cloud billing export.",
> "x": ["Search", "Ads", "Infra", "Data"], "y": [182, 96, 240, 131], "ylabel": "$ thousands"},
> "off_brand": {"note": ""}},
> ]
>
>
> def draw(spec: dict, dark: bool, path: Path) -> None:
> """Render one chart spec to a 900 x 560 PNG."""
> plt.style.use("dark_background" if dark else "default")
> kind, x, y, colors = spec["kind"], spec["x"], spec["y"], spec.get("colors", BRAND)
> fig = plt.figure(figsize=(9, 5.6), dpi=100)
> ax = fig.add_subplot(projection="3d" if kind == "bar3d" else None)
> if kind == "bar":
> ax.bar(x, y, color=colors if "colors" in spec else colors[0])
> elif kind == "hbar":
> ax.barh(x, y, color=colors[0])
> elif kind in ("line", "area"):
> ax.plot(x, y, marker="o", lw=3, color=colors[0], label=spec.get("label"))
> if kind == "area":
> ax.fill_between(range(len(y)) if isinstance(x[0], str) else x, y, alpha=0.3, color=colors[0])
> if spec.get("y_extra"):
> ax.plot(x, spec["y_extra"], marker="o", lw=3, color=colors[1], label=spec["label_extra"])
> ax.legend(loc="upper left")
> elif kind in ("pie", "donut"):
> ring = {"width": 0.45} if kind == "donut" else None
> ax.pie(y, labels=[f"{name}\n{v}%" for name, v in zip(x, y)], colors=colors,
> startangle=90, counterclock=False, wedgeprops=ring, textprops={"fontsize": 13})
> elif kind == "bar3d":
> ax.bar3d(range(len(y)), 0, 0, 0.6, 0.6, y, color=colors[0], shade=True)
> ax.set_xticks([i + 0.3 for i in range(len(x))], x)
> ax.set_yticks([])
> ax.set_zlabel(spec["ylabel"])
> ax.view_init(elev=28, azim=-58)
> if spec.get("y2"): # a second value axis on the right
> ax2 = ax.twinx()
> ax2.plot(x, spec["y2"], marker="s", lw=3, color=colors[1], label=spec["label2"])
> ax2.set_ylim(*spec["ylim2"])
> ax2.set_ylabel(spec["label2"])
> ax.legend(loc="upper left")
> ax2.legend(loc="lower right")
> value_lim = ax.set_xlim if kind == "hbar" else ax.set_ylim
> value_label = ax.set_xlabel if kind == "hbar" else ax.set_ylabel
> if kind not in ("pie", "donut", "bar3d"):
> if spec.get("log"):
> ax.set_yscale("log")
> else:
> value_lim(*spec.get("ylim", (0, None)))
> value_label(spec.get("ylabel", ""))
> if spec.get("xlabel"):
> ax.set_xlabel(spec["xlabel"])
> if spec.get("invert"):
> ax.invert_yaxis()
> if spec.get("hide_ticks"):
> ax.tick_params(axis="y", left=False, labelleft=False)
> ax.spines[["top", "right"]].set_visible(bool(spec.get("y2")))
> if spec.get("center_title"):
> fig.suptitle(spec["title"], fontsize=16, fontweight="bold")
> else:
> fig.suptitle(spec["title"], fontsize=16, fontweight="bold", x=0.02, ha="left")
> fig.text(0.02, 0.015, spec["note"], fontsize=10, alpha=0.75)
> fig.savefig(path)
> plt.close(fig)
>
>
> def main() -> None:
> OUT.mkdir(exist_ok=True)
> labels = []
> for case in CASES:
> for variant in ("honest", "misleading", "off_brand"):
> if variant == "honest" or variant in case:
> labels.append({"file": f"{OUT}/{case['id']}__{variant}.png", "case": case["id"], "set": case["set"],
> "variant": variant,
> "pattern": case.get("pattern") if variant == "misleading" else None,
> "brand": case.get("brand") if variant == "off_brand" else None})
> # The answer key is written before any chart exists, and long before any API call.
> (OUT / "labels.json").write_text(json.dumps(labels, indent=2))
> for case in CASES:
> for label in (row for row in labels if row["case"] == case["id"]):
> spec = {**case["honest"], **case.get(label["variant"], {})}
> draw(spec, case.get("dark", False), Path(label["file"]))
> print(f"Wrote {len(labels)} charts and {OUT}/labels.json")
>
>
> if __name__ == "__main__":
> main()
> ```
> **Draw the charts, then check two of them**
>
> ```bash theme={null}
> python make_charts.py
> python chart_check.py charts/spend_signups__misleading.png charts/spend_signups__honest.png charts/plan_colors__off_brand.png
> ```
> **Observed output**
>
> ```text theme={null}
> charts/spend_signups__misleading.png
> design: misleading (line_or_area 0.99, overall 1.45 of 3)
> 0.99 Two value axes with mismatched scales
> Why: Each axis was scaled separately, so the two lines track each
> other by construction. The overlap suggests a link the data does not
> show.
> brand: conforms
> 1,842 input tokens, 0.89 s, request 46c483c4-00d5-4a1b-8045-e55c5b4b6809
> charts/spend_signups__honest.png
> design: clear (line_or_area 1.00, overall 0.06 of 3)
> brand: conforms
> 1,842 input tokens, 1.99 s, request dfaca84f-e9f2-42d3-bb6c-6d150e43a3c1
> charts/plan_colors__off_brand.png
> design: clear (bar 1.00, overall 0.05 of 3)
> brand: off-brand
> 1.00 Data colors are outside the brand palette
> Fix: Redraw the data in dark teal, light teal, orange, or rust.
> 1,842 input tokens, 0.46 s, request abb5a6ed-c55c-43d8-a843-dc9f02cf5198
> ```
### Scoring
`evaluate.py` runs `chart_check.py` over a set from `charts/labels.json` and counts, separately for design and for brand: planted problems caught (the planted check is in `flagged` or `brand_flagged`), other charts passed (`clear` for design, `conforms` for brand), and false flags (any flag on a chart where nothing of that kind was planted). Checks listed in `EXPERIMENTAL` get their own row so they cannot inflate or sink the design number. It also prints the score of a checker that sends every chart to `review`. That checker looks careful and does nothing: it catches no problem and passes no chart. Then it lists every miss.
> **evaluate.py**
>
> ```python evaluate.py theme={null}
> """Score chart_check.py against the answer key in charts/labels.json.
>
> python evaluate.py --set dev,control # tune cutoffs and wording on these
> python evaluate.py --set holdout # once, after everything is frozen
> """
>
> from __future__ import annotations
>
> import argparse
> import json
> import os
> import sys
>
> import httpx
>
> import chart_check
> from checklist import EXPERIMENTAL
>
> PRICE_PER_M_INPUT = 0.02 # USD per million input tokens on pplx-decider-latest; output is free
> EMPTY = {"planted": 0, "caught": 0, "fine": 0, "passed": 0, "false_flags": 0}
>
>
> def tally(labels: list[dict], records: dict[str, dict]) -> tuple[dict, list[str]]:
> """Count catches, passes, and false flags per set for design and brand, and describe every miss.
>
> Planted experimental checks are counted on their own row, not in the design row.
> """
> table: dict[str, dict] = {}
> misses = []
> for lab in labels:
> r = records[lab["file"]]
> rows = table.setdefault(lab["set"], {"design": dict(EMPTY), "brand": dict(EMPTY)})
> for kind, planted, flagged, good in (("design", lab["pattern"], r["flagged"], "clear"),
> ("brand", lab["brand"], r["brand_flagged"], "conforms")):
> row, got = rows[kind], r["label"] if kind == "design" else r["brand"]
> if planted in EXPERIMENTAL:
> row = rows.setdefault("exper.", dict(EMPTY))
> if planted:
> row["planted"] += 1
> row["caught"] += planted in flagged
> if planted not in flagged:
> misses.append(f"{lab['file']}: {planted} {r['probabilities'][planted]:.2f}, {kind} {got}")
> else:
> row["fine"] += 1
> row["passed"] += got == good
> row["false_flags"] += len(flagged)
> if got != good:
> top = ", ".join(f"{n} {r['probabilities'][n]:.2f}" for n in flagged) or f"overall {r['overall']:.2f}"
> misses.append(f"{lab['file']}: expected {kind} {good}, got {got} ({top})")
> return table, misses
>
>
> def main() -> int:
> parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
> parser.add_argument("--set", default="dev,control", help="comma-separated: dev, control, holdout")
> parser.add_argument("--out", default="results.jsonl", help="append one JSON line per image")
> args = parser.parse_args()
>
> api_key = os.environ.get("PERPLEXITY_API_KEY")
> if not api_key:
> print("Set PERPLEXITY_API_KEY first.", file=sys.stderr)
> return 2
> sets = args.set.split(",")
> with open("charts/labels.json", encoding="utf-8") as handle:
> labels = [lab for lab in json.load(handle) if lab["set"] in sets]
>
> records: dict[str, dict] = {}
> with httpx.Client(headers={"Authorization": f"Bearer {api_key}"}, timeout=90.0) as client:
> for lab in labels:
> try:
> records[lab["file"]] = chart_check.check_file(client, lab["file"])
> except (RuntimeError, httpx.HTTPError) as error:
> print(f"{lab['file']}: Decisions API failed: {error}", file=sys.stderr)
> return 1
> r = records[lab["file"]]
> print(f"{r['label']:<11}{r['brand']:<10}{lab['file']:<44}{r['input_tokens']:>6,} tokens {r['seconds']:>5.2f} s")
> with open(args.out, "a", encoding="utf-8") as handle:
> handle.writelines(json.dumps(r) + "\n" for r in records.values())
>
> table, misses = tally(labels, records)
> print(f"\n{'set':<9}{'check':<8}{'planted caught':>16}{'others passed':>15}{'false flags':>13}")
> for name, rows in table.items():
> for kind, row in rows.items():
> caught = f"{row['caught']} of {row['planted']}" if row["planted"] else "n/a"
> print(f"{name:<9}{kind:<8}{caught:>16}{row['passed']:>10} of {row['fine']:<3}{row['false_flags']:>12}")
> planted = sum(row["planted"] for rows in table.values() for row in rows.values())
> fine = sum(row["fine"] for rows in table.values() for row in rows.values())
> # The bar to beat: a checker that sends every chart to review looks careful and does nothing.
> print(f"always-review baseline: 0 of {planted} planted caught, 0 of {fine} others passed")
> for miss in misses:
> print("miss:", miss)
> tokens = sum(r["input_tokens"] for r in records.values())
> print(f"\n{len(records)} charts, {tokens:,} input tokens, ${tokens / 1e6 * PRICE_PER_M_INPUT:.5f}")
> return 0
>
>
> if __name__ == "__main__":
> sys.exit(main())
> ```
Tune on the development set and the controls. Change the criteria or the cutoffs, rerun, repeat. When you are done, freeze `checklist.py` and `chart_check.py`, then score the held-out set once.
> **Score the development set, then the held-out set**
>
> ```bash theme={null}
> python evaluate.py --set dev,control
> python evaluate.py --set holdout
> ```
### Results
Measured on 2026-10-19 against `pplx-decider-latest`, with the files on this page.
Two changes came from the development set and the controls, and the cutoffs did not change. The first version of the source rule asked for a line that starts with "Source:". The test charts put the source at the end of a note, and the model gave that rule 0.66 to 0.94 on 8 of 23 conforming charts, a fair reading of the rule as written. The rule now asks for "Source:" anywhere in the note. The first version of the title rule had no answer for a chart with no title, and the model gave it about 0.6 on untitled charts; the rule now says an untitled chart is fine, and the no-title control scores 0.04. After those two edits the files were frozen and the held-out set was scored once.
The design checks and cutoffs come from an earlier version of this checker that was tested on different renderings of the four held-out design cases. Nothing in the design checks was changed in response to them. The three held-out brand cases are new.
| Set | Check | Planted problem caught | Other charts passed | False flags |
| - | - | - | - | - |
| Development (12 pairs) | Design | 8 of 8 (planted check 0.81 to 1.00) | 15 of 15 (highest check 0.18) | 0 |
| | Brand | 3 of 3 (each 1.00) | 21 of 21 (highest rule 0.05) | 0 |
| | Experimental | 1 of 1 (0.65) | n/a | 0 |
| Controls (3 honest charts) | Design | n/a | 3 of 3 (every check 0.04 or lower) | 0 |
| | Brand | n/a | 3 of 3 | 0 |
| Held out (7 pairs) | Design | 3 of 3 | 10 of 10 | 0 |
| | Brand | 3 of 3 (each 1.00) | 11 of 11 | 0 |
| | Experimental | 0 of 1 (0.02) | n/a | 0 |
| Always-review baseline, all sets | Both | 0 of 19 | 0 of 63 | 0 |
> **Observed output, held-out run**
>
> ```text theme={null}
> clear conforms charts/h_margin_bars__honest.png 1,842 tokens 0.70 s
> misleading conforms charts/h_margin_bars__misleading.png 1,842 tokens 0.42 s
> clear conforms charts/h_dual_climate__honest.png 1,842 tokens 1.48 s
> misleading conforms charts/h_dual_climate__misleading.png 1,842 tokens 0.48 s
> clear conforms charts/h_pie_sum__honest.png 1,842 tokens 0.44 s
> clear conforms charts/h_pie_sum__misleading.png 1,842 tokens 0.43 s
> clear conforms charts/h_title_claim__honest.png 1,842 tokens 0.43 s
> misleading conforms charts/h_title_claim__misleading.png 1,842 tokens 0.50 s
> clear conforms charts/h_signup_colors__honest.png 1,842 tokens 0.43 s
> clear off-brand charts/h_signup_colors__off_brand.png 1,842 tokens 0.45 s
> clear conforms charts/h_share_title__honest.png 1,842 tokens 0.43 s
> clear off-brand charts/h_share_title__off_brand.png 1,842 tokens 0.43 s
> clear conforms charts/h_cost_source__honest.png 1,842 tokens 0.43 s
> clear off-brand charts/h_cost_source__off_brand.png 1,842 tokens 1.82 s
>
> set check planted caught others passed false flags
> holdout design 3 of 3 10 of 10 0
> holdout brand 3 of 3 11 of 11 0
> holdout exper. 0 of 1 0 of 0 0
> always-review baseline: 0 of 7 planted caught, 0 of 21 others passed
> miss: charts/h_pie_sum__misleading.png: parts_dont_sum 0.02, design clear
>
> 14 charts, 25,788 input tokens, $0.00052
> ```
The held-out request ids, in the order above, are `5353438d-fcf5-4f6e-8734-b85d9046360e`, `b54231cf-364c-4810-8ed5-4a33649ff178`, `b0b8f70d-8837-48c3-86f0-6b8a2e3ef9b8`, `a5f76ad8-bfe6-4c87-b243-2c4bb61adfc1`, `d23e9405-223a-49fa-b3b9-0d7a7849db6b`, `bba66963-686b-43e6-8823-143bf9a10c08`, `e93c58e5-798b-4059-a923-df2a1f7b961a`, `bc8cb4f2-63fa-4846-8a11-7c60c93c0795`, `80ecf435-9364-4ab8-8499-846c3b6d4eb0`, `d7138a6f-979e-4c68-b2e7-15980545b016`, `db49cf9a-3b21-4167-89d1-041516554b71`, `6097cc2c-a3e3-43d1-92c5-4d2be0d31a76`, `48d1178c-2b17-4297-9863-b38ebeeab2cf`, `44188ec6-beac-490e-be1c-11e9f0aaeb22`. `results.jsonl` has every request id, probability, and token count from your own runs.
**The experimental check.** `parts_dont_sum` asks whether pie labels add to more than 100%. The development pie, whose three labels add to 130%, scored 0.65, just over `FLAG`. The held-out donut, whose four labels add to 123%, scored 0.02 and was labeled `clear`. Adding numbers across labels is the weakest check on the list by a wide margin, which is why it is in `EXPERIMENTAL` and has its own row. Keep it visible, do not act on it alone, and if pie totals matter to you, check the source data in code.
**Run to run.** Two development runs on 2026-10-19 with the final files returned identical probabilities for all 27 charts. Do not count on that: the [Decisions API docs](https://docs.perplexity.ai/docs/decisions/quickstart.md#model-names) say `pplx-decider-latest` can move to a newer decision model, and you need to revalidate the cutoffs when it does. Your own charts drawn with the same script can also render slightly differently on another machine, which can move a probability near a cutoff.
**Cost and speed.** Every test chart was 1,842 input tokens. At \$0.02 per million input tokens, with output tokens free, scoring all 41 charts once costs about \$0.0015, and a chart costs about \$0.00004. Across the 41 requests in the development and held-out runs, the median was 0.45 seconds and 9 in 10 finished within 0.54 seconds. The three slowest took 1.0, 1.5, and 1.8 seconds, so plan for an occasional slow request.
### Charts nobody drew for this page
The 41 test charts come from one script, so they show the checks work on clean, single-problem charts. They do not show what happens on charts drawn by other tools. After the held-out run, the frozen files were run with `--no-brand` on 11 screenshots from public sites: four sample dashboards from Tableau Public, a life expectancy chart from Our World in Data, five examples from the Wikipedia article on misleading graphs (the truncated bar graph and its full-scale version, a 3D pie and its flat version, and a two-bar chart with a cut axis), and a Federal Reserve chart of military spending with two value axes. None had been seen by this version of the checker before.
* Every honest chart was `clear` except the Tableau stocks dashboard, which went to `review` because the model gave its chart type a confidence of 0.42. Dashboards mix chart kinds, and that rule is doing what it is for.
* The truncated bar graph was `misleading` at 0.98, its full-scale twin `clear` at 0.00. The 3D pie was `misleading` at 1.00, the flat pie `clear`. The two-bar chart was `misleading` at 0.95.
* The military spending chart, the kind of two-axis chart people argue about, came back at 0.61 for `dual_axis_mismatch` with 11 questions and at 0.45 with the brand guide and its three questions in the same request. Same image, same model, `misleading` one way and `review` the other. That is the grouping effect from the request section, measured, and it is why you freeze the checklist before you trust a cutoff and rescore everything when you add a question.
The no-title fix to the brand rule came from these screenshots, so they are not a clean held-out set for that rule. They are a clean set for the design checks: 7 of 7 honest charts passed (6 `clear`, 1 `review`), 4 of 4 misleading charts caught, 0 false flags. Label a sample of your own charts and run `evaluate.py` on them before you trust a cutoff on your dashboards.
## Run it on a schedule
The practical shape of this recipe is a sweep, not a one-off. Export every published dashboard as an image each night, run `chart_check.py` over the folder, and send each `misleading` or `review` result to the chart's owner with its `Why:` lines. The dashboards above cost about 2,500 input tokens each, so a chart costs about \$0.00005; at the default 10 requests per second, that is 36,000 charts an hour for under \$2. A GitHub Action that checks every PNG a notebook produces on a pull request is the same loop with a different trigger. `results.jsonl` already has everything a report needs: both labels, every probability, and a request id per chart.
## Run the tests
The tests cover the request shape with and without the brand guide, the tile budget, transparent images, chart-type gating, the review rules including an uncertain chart type, brand and design labels staying separate, the printed reasons, the `429` retry with both forms of `Retry-After`, and the scoring with an experimental check. They make no network calls and need no key.
> **test\_chart\_check.py**
>
> ```python test_chart_check.py theme={null}
> """Offline checks for chart_check.py and evaluate.py. No network, no API key.
>
> Run: python -m pytest -q
> """
>
> from __future__ import annotations
>
> import io
>
> import httpx
> from PIL import Image
>
> import chart_check as cc
> from checklist import BRAND_CHECKS, CHECKS
> from evaluate import tally
>
>
> def png(w: int, h: int) -> bytes:
> buf = io.BytesIO()
> Image.new("RGB", (w, h), "white").save(buf, format="PNG")
> return buf.getvalue()
>
>
> def answers(chart_type: str = "bar", overall: float = 0.2, **probs: float) -> dict:
> """A response's `answers` in the documented shape. Values are test inputs, not model output."""
> a = {"chart_type": {"type": "choice", "choice": chart_type, "confidence": 0.9},
> "overall": {"type": "score", "score": overall}}
> for name in [*CHECKS, *BRAND_CHECKS]:
> a[name] = {"type": "noul", "noul": probs.get(name, 0.01)}
> return a
>
>
> def test_request_has_chart_type_design_checks_brand_rules_and_overall():
> body = cc.build_request(png(64, 64))
> assert body["model"] == "pplx-decider-latest"
> assert list(body["questions"]) == ["chart_type", *CHECKS, *BRAND_CHECKS, "overall"]
> assert body["state"]["chart"]["image_url"]["url"].startswith("data:image/png;base64,")
> assert "brand_guide" in body["state"]
> without = cc.build_request(png(64, 64), brand=False)
> assert list(without["questions"]) == ["chart_type", *CHECKS, "overall"] and "brand_guide" not in without["state"]
>
>
> def test_large_images_shrink_to_the_tile_budget_and_small_ones_do_not():
> w, h = Image.open(io.BytesIO(cc.fit_to_tiles(png(3000, 2000)))).size
> assert round(w / 32) * round(h / 32) <= 2048
> assert Image.open(io.BytesIO(cc.fit_to_tiles(png(900, 560)))).size == (900, 560)
>
>
> def test_transparent_background_becomes_white_not_black():
> buf = io.BytesIO()
> Image.new("RGBA", (64, 64), (0, 0, 0, 0)).save(buf, format="PNG") # black under a transparent alpha
> sent = Image.open(io.BytesIO(cc.fit_to_tiles(buf.getvalue())))
> assert sent.mode == "RGB" and sent.getpixel((5, 5)) == (255, 255, 255)
>
>
> def test_a_design_check_counts_only_on_the_chart_type_it_applies_to():
> bars = cc.judge(answers("bar", truncated_baseline=0.95))
> assert bars.label == "misleading" and bars.flagged == ["truncated_baseline"]
> line = cc.judge(answers("line_or_area", truncated_baseline=0.95))
> assert line.label == "clear" and line.suppressed == ["truncated_baseline"]
>
>
> def test_borderline_probability_or_high_overall_score_means_review():
> assert cc.judge(answers("bar", title_overclaims=0.5)).label == "review"
> assert cc.judge(answers("pie_or_donut", overall=1.8)).label == "review"
> assert cc.judge(answers("pie_or_donut", overall=0.3)).label == "clear"
>
>
> def test_brand_is_judged_separately_from_design_and_can_be_skipped():
> v = cc.judge(answers("bar", off_brand_colors=0.9))
> assert (v.label, v.brand, v.brand_flagged) == ("clear", "off-brand", ["off_brand_colors"])
> v = cc.judge(answers("bar", truncated_baseline=0.9, missing_source=0.5))
> assert (v.label, v.brand) == ("misleading", "review")
> design_only = {k: v for k, v in answers("bar").items() if k not in BRAND_CHECKS}
> assert cc.judge(design_only).brand == "skipped"
>
>
> def test_a_guessed_chart_type_sends_the_chart_to_review():
> a = answers("bar")
> a["chart_type"]["confidence"] = 0.3
> assert cc.judge(a).label == "review"
>
>
> def test_each_finding_prints_its_probability_then_its_reason():
> name, (short, reason, _, _) = next(iter(CHECKS.items()))
> lines = cc.explain([name], CHECKS, {name: 0.97}, "Why: ")
> assert lines[0].strip() == f"0.97 {short}" or lines[0].strip().startswith(f"0.97 {short}")
> assert " ".join(line.strip() for line in lines[1:]) == f"Why: {reason}"
>
>
> def test_decide_waits_out_a_429_then_returns_the_answer(monkeypatch):
> replies = iter([httpx.Response(429, headers={"Retry-After": "0"}),
> httpx.Response(200, json={"answers": answers(), "usage": {"input_tokens": 1}})])
> monkeypatch.setattr(cc.time, "sleep", lambda s: None)
> client = httpx.Client(transport=httpx.MockTransport(lambda request: next(replies)))
> data, response = cc.decide(client, {"model": cc.MODEL})
> assert response.status_code == 200 and data["usage"]["input_tokens"] == 1
>
>
> def test_retry_after_accepts_seconds_or_an_http_date():
> assert cc.retry_after("2") == 2.0 and cc.retry_after(None) == 1.0
> assert cc.retry_after("Wed, 21 Oct 2015 07:28:00 GMT") == 0.0 # a past date means no wait
>
>
> def test_tally_counts_a_missed_pattern_and_a_false_flag():
> labels = [{"file": "a", "set": "dev", "pattern": "parts_dont_sum", "brand": None},
> {"file": "b", "set": "dev", "pattern": None, "brand": None}]
> records = {"a": vars(cc.judge(answers("pie_or_donut", parts_dont_sum=0.1))),
> "b": vars(cc.judge(answers("bar", truncated_baseline=0.9)))}
> table, misses = tally(labels, records)
> assert table["dev"]["exper."] == {"planted": 1, "caught": 0, "fine": 0, "passed": 0, "false_flags": 0}
> assert table["dev"]["design"] == {"planted": 0, "caught": 0, "fine": 1, "passed": 0, "false_flags": 1}
> assert table["dev"]["brand"]["passed"] == 2 and len(misses) == 2
> ```
> **Run the tests**
>
> ```bash theme={null}
> python -m pytest -q
> ```
## What this is not
* **It does not check the data.** A chart can start at zero, label every axis, and still plot the wrong numbers.
* **It cannot see what was left out.** A chart that starts the date range at a convenient low point looks fine as an image. That is a question about the data behind the chart, not the pixels.
* **It does not judge intent.** A bar chart of body temperature with an axis starting at 95 °F is a defensible choice. `misleading` means a pattern is present; a person decides what to do about it.
* **The reasons are yours, not the model's.** Each `Why:` and `Fix:` line is the sentence you wrote in `checklist.py` for that check. It explains the check, not this particular chart.
* **It does not check rendering.** Clipped text, overlapping cards, a button pushed off screen, or a blank page are defects in how a page was drawn, and they need different questions than the ones here. This recipe assumes the chart rendered as its author intended and asks whether that rendering tells the truth.
* **It does not replace a brand review.** Three rules about colors, a source line, and a title cover what is easy to see in an image. Fonts, logo use, and tone of voice need a person. The color rule catches blue where teal belongs, not a teal two shades off; an exact palette check is a pixel count you can do in Pillow without a model.
## Adapt it
* **Your brand guide.** Replace `BRAND_GUIDE` and `BRAND_CHECKS` with your rules, add an honest/off-brand pair for each to `make_charts.py`, and rerun the development set. Describe colors by name as well as by look ("dark teal, not blue"), because the model reads the image, not your hex codes. Say what happens when a rule does not apply, as the title rule does for untitled charts.
* **Your design checks.** Add an entry to `CHECKS` and to `APPLIES_TO`, add a pair to `make_charts.py`, and rerun. Adding a question changes the numbers for the others, so rescore everything.
* **Your cutoffs.** Label a few hundred of your own charts, run `evaluate.py` on them, and move `FLAG` and `BORDERLINE`. Leave margin around where your charts cluster; a cutoff on top of a cluster turns small movement into a different label.
* **Context.** Pass `context` to `build_request`, for example "Quarterly board deck, finance team", to give the model the setting the chart appears in.
* **The other decision model.** Set `MODEL = "pplx-decider-flash-latest"` to try the smaller model. The docs recommend one question per request for Flash by default, and this recipe groups fourteen, so measure it on your own labeled charts first. The results on this page are for `pplx-decider-latest` only.
## Troubleshooting
Every Decisions API error prints as `: Decisions API failed: (x-request-id )`.
* `Set PERPLEXITY_API_KEY first.` means the key is not exported in this terminal. Run the `export` line again.
* `400` with `Invalid model` means `MODEL` is not a name the API accepts. Use `pplx-decider-latest`.
* `401` is a missing or invalid key.
* `504` after about a minute usually means an image over the tile budget. `fit_to_tiles` prevents that for files sent through `check_file`; if you build requests another way, resize first.
* `ModuleNotFoundError: No module named 'checklist'` means `checklist.py` is not in the directory you run from. Save it next to `chart_check.py`, and run both `chart_check.py` and `evaluate.py` from that directory.
* `No matching distribution found for matplotlib==3.11.2` during install means the virtual environment uses Python older than 3.11. Remove `.venv` and create it again with a newer Python, for example `python3.12 -m venv .venv`.