> ## Documentation Index
> Fetch the complete documentation index at: https://docs.perplexity.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Search Your Docs Folder with pplx-embed-v2, Dense and Contextual

> Index a folder of Markdown files with pplx-embed-v2 in dense and contextual mode, and store them as int8 vectors. Score approaches against your own questions. Query the 9b index with the 0.6b model.

You have a folder of Markdown files and want to type a question and get the section that answers it. This recipe indexes the folder two ways with one model, shows you the two answers side by side, and measures which way finds the right section on questions you wrote. On the FastAPI documentation, the contextual index put the answer in the top 5 for 28 of 30 questions and the dense index for 23.

The tool is one 443-line Python file with three commands: `index` a folder, `ask` it a question, and `eval` it against labeled questions. `pplx-embed-v2-context-9b` embeds your chunks either alone or with the rest of their page through one endpoint. The demonstration corpus is the [FastAPI documentation](https://github.com/fastapi/fastapi), 144 files and 2,683 chunks. The tool reads any folder of `.md` files, with one caveat the chunking section explains: the character budgets it uses for the API's token caps were tuned for this corpus.

The v2 models serve both input modes from one checkpoint. Send a flat list of strings and each text is embedded on its own: dense mode, one vector per text. Send a list of lists, where each inner list is the chunks of one document in reading order, and each chunk vector reflects the rest of its document: contextual mode, one vector per chunk, so a section headed "Run it" or "Check it" carries what the page is about. Every request names its purpose with `input_type`, `document` for anything you index and `query` for a question. The two sizes of the model share one vector space, and both can return shorter vectors, which gives the tool its four configurations: dense at 2048 dimensions, contextual at 2048, contextual at 1024, and contextual at 1024 with the questions embedded by the 0.6b model.

The research behind the contextual mode, [Contextual Embeddings Beyond the Gold Chunk](https://arxiv.org/abs/2610.38854), measures retrieval by the share of questions whose answer chunk lands in the top K results. The `eval` command reports the same idea as recall\@k on your questions. The weights are published on Hugging Face as [pplx-embed-v2-context-9b](https://huggingface.co/perplexity-ai/pplx-embed-v2-context-9b) and [pplx-embed-v2-context-0.6b](https://huggingface.co/perplexity-ai/pplx-embed-v2-context-0.6b); the API serves them as `pplx-embed-v2-context-9b` and `pplx-embed-v2-context-0.6b`.

Here is what one `ask` looks like on the FastAPI docs, with the default two configurations, dense and contextual at 2048 dimensions:

````text theme={null}
$ python docsearch.py ask "How do I make a path parameter accept values with slashes in them?"

dense-2048: dense index at 2048 dimensions, question embedded by pplx-embed-v2-context-9b
  1. tutorial/path-params.md > Path parameters containing paths
     ## Path parameters containing paths Let's say you have a *path operation* with a path `/files/{file_path}`. But you need `file_path` itself to contain a *path*,
  2. advanced/path-operation-advanced-configuration.md > Path Operation Advanced Configuration
     # Path Operation Advanced Configuration
  3. tutorial/path-params-numeric-validations.md > Declare metadata
     A path parameter is always required as it has to be part of the path. Even if you declared it with `None` or set a default value, it would not affect anything,

context-2048: context index at 2048 dimensions, question embedded by pplx-embed-v2-context-9b
  1. tutorial/path-params.md > Path convertor
     ### Path convertor Using an option directly from Starlette you can declare a *path parameter* containing a *path* using a URL like: ``` /files/{file_path:path}
  2. tutorial/path-params.md > OpenAPI support
     ### OpenAPI support OpenAPI doesn't support a way to declare a *path parameter* to contain a *path* inside, as that could lead to scenarios that are difficult t
  3. tutorial/path-params.md > Path Parameters
     # Path Parameters You can declare path "parameters" or "variables" with the same syntax used by Python format strings: ```Python hl_lines="6-7" from fastapi imp
````

Both configurations go to `tutorial/path-params.md`. Dense mode puts the parent section first, "Path parameters containing paths", which states the problem and hands off to the next section; the section that answers it, "Path convertor" with `/files/{file_path:path}`, is its sixth result. Contextual mode puts "Path convertor" first and fills the next two places with neighboring sections of the same page. The rest of this page builds the tool, runs it, and shows where the configurations differ.

## What you need

* Python 3.12 or newer on macOS or Linux (the pinned numpy release needs 3.12). The shell commands use Unix syntax.
* A Perplexity API key with API credits. Indexing the FastAPI docs sends the corpus three times; the recorded run used 1,815,960 tokens and cost $0.2179 at $0.12 per million tokens for the 9b model. The 0.6b model is $0.02 per million. Both rates were confirmed by direct probes while this page was written, and the indexing total agrees: $0.2179 for 1,815,960 tokens at \$0.12 per million. The [pricing page](https://docs.perplexity.ai/docs/getting-started/pricing.md#embeddings-api-pricing) is the reference for current rates. Each `ask` or `eval` run embeds only the questions.
* httpx and numpy, installed in the next step. The recipe calls the REST endpoint directly because the Perplexity Python library release current at the time of writing (0.46.1) has no `input_type` parameter and does not accept nested input on `embeddings.create`.
* `curl`, `tar`, and `python3` on the PATH to fetch and prepare the sample corpus, or any folder of Markdown files you already have.

## Set up

Everything lives in one directory. You create the four files from this page; the corpus is fetched with one command.

```text theme={null}
docsearch/
├── requirements.txt       # three pinned dependencies
├── docsearch.py           # index, ask, eval
├── test_docsearch.py      # offline tests, no API key
├── questions.jsonl        # 30 labeled questions over the FastAPI docs
├── fastapi-docs/          # the corpus, fetched below (144 .md files)
├── index/                 # written by `index`: vectors.npz, chunks.json, models.json
└── .venv/                 # virtual environment from the install step
```

> [accordion] requirements.txt
>
> ```text requirements.txt theme={null}
> httpx==0.28.1
> numpy==2.5.4
> pytest==9.1.1
> ```

Create the directory, save `requirements.txt` in it, then create a virtual environment, install the packages, and export your key.

> [accordion] Install and set the key
>
> ```bash theme={null}
> mkdir docsearch && cd docsearch
> # save requirements.txt here, then continue
> python3 -m venv .venv
> source .venv/bin/activate
> python -m pip install -r requirements.txt
> export PERPLEXITY_API_KEY="your-api-key-here"
> ```

## Fetch a corpus

The sample corpus is the English documentation of FastAPI 0.115.0, from the release tarball. The FastAPI pages do not hold their code examples inline; a line like `{!../../../docs_src/path_params/tutorial001.py!}` pulls the file in when the site is built. A search index built from the raw pages would have 416 chunks that hold an include path where the code should be, so the second step inlines every include before the folder is indexed:

```bash theme={null}
curl -L https://github.com/fastapi/fastapi/archive/refs/tags/0.115.0.tar.gz | tar xzf -
python3 - <<'EOF'
import re
from pathlib import Path

docs = Path("fastapi-0.115.0/docs/en/docs")
include = re.compile(r"\{!>? ?(\S+?)(?:\[ln:(\d+)-(\d+)\])?!\}")
for page in docs.rglob("*.md"):
    lines = []
    for line in page.read_text().splitlines():
        if m := include.fullmatch(line.strip()):
            source = (docs / m.group(1)).read_text().splitlines()
            lines += source[int(m.group(2)) - 1 : int(m.group(3))] if m.group(2) else source
        else:
            lines.append(line)
    page.write_text("\n".join(lines) + "\n")
EOF
mv fastapi-0.115.0/docs/en/docs fastapi-docs && rm -rf fastapi-0.115.0
```

The tarball unpacks to about 30 MB. The Python step replaces each include line with the referenced file, or with the `[ln:a-b]` line range when the include names one, and accepts the `{!> ...!}` form the docs also use. The last line keeps the English docs folder, images and stylesheets included, and removes the rest. The tool reads only the `.md` files: 144 of them, about 2.2 million characters once the code is inlined, including a tutorial, an advanced guide, deployment pages, reference pages, and a 445 KB release-notes file that is longer than a single contextual document is allowed to be. That last file is why the tool knows how to split a document. This preprocessing is specific to FastAPI; `docsearch.py` reads plain Markdown and knows nothing about includes.

To use your own docs instead, skip this step and point `index` at your folder.

## Write the tool: docsearch.py

One file, five parts. Each part is below with what it does; the complete file is at the end of the page, in two blocks you append in order.

### Name the configurations and chunk a folder of Markdown files

A `Config` is one column of the evaluation: how the corpus was embedded (`mode` and `dimensions`) and which model embeds the questions. The four in `CONFIGS` share three indexes, because the fourth reuses the contextual 1024 index and only changes the query model. The index is always built with `INDEX_MODEL`, the 9b model; `SMALL_MODEL` appears only as a query model. Add a configuration by adding a line.

A chunk is one heading section, heading included, so every chunk starts with its own title. Sections longer than about 1,500 characters are split at blank lines outside code fences, then at line breaks, so no chunk exceeds `CHUNK_CHARS`; a `#` inside a code fence is not a heading. A document is one file in reading order, which is the order the contextual mode needs. The API caps a text or a contextual document in tokens, and `DOC_CHARS` is a character budget for that cap, tuned to this corpus: the first draft used 60,000 and the first contextual request came back as a 400 for a 19,689-token document. If your corpus has fewer characters per token (CJK text, minified code, long link lists), lower it or count tokens with the published tokenizer. A file over the budget becomes consecutive documents named `path#2`, `path#3`.

> [accordion] docsearch.py: imports, constants, and chunking
>
> ```python theme={null}
> """Search a folder of Markdown files with pplx-embed-v2: the same model in dense and contextual mode, side by side.
>
> Usage:
>   python docsearch.py index <folder>              chunk every .md file, embed it three ways, write index/
>   python docsearch.py ask "<question>" [-k 3] [--configs NAME ...]   top chunks per configuration
>   python docsearch.py eval questions.jsonl         recall@k and MRR@100 for each configuration on labeled questions
> """
>
> import argparse
> import base64
> import json
> import os
> import re
> import sys
> import time
> from dataclasses import asdict, dataclass
> from pathlib import Path
>
> import httpx
> import numpy as np
>
> API_URL = "https://api.perplexity.ai/v1/embeddings"
> # One checkpoint per size serves both input modes. The index is always built with the 9b model; the 0.6b model shares
> # its vector space at 1024 dimensions, so it can embed questions against a 9b index built at 1024.
> INDEX_MODEL = "pplx-embed-v2-context-9b"
> SMALL_MODEL = "pplx-embed-v2-context-0.6b"
>
>
> @dataclass(frozen=True)
> class Config:
>     """One column of the evaluation: how the corpus was embedded and which model embeds the questions."""
>
>     name: str
>     mode: str  # "dense": every chunk embedded alone. "context": each document's chunks embedded together.
>     dimensions: int
>     query_model: str
>
>     @property
>     def index_key(self) -> str:
>         return f"{self.mode}-{self.dimensions}"
>
>
> CONFIGS = [
>     Config("dense-2048", "dense", 2048, INDEX_MODEL),
>     Config("context-2048", "context", 2048, INDEX_MODEL),
>     Config("context-1024", "context", 1024, INDEX_MODEL),
>     Config("context-1024-small", "context", 1024, SMALL_MODEL),
> ]
>
> # A chunk is one heading section. Longer sections are split at blank lines into pieces of about this size, and a single
> # line longer than this (an unwrapped paragraph, a minified file, an inline image) is cut into pieces of this size.
> CHUNK_CHARS = 1500
> # The API's caps are in tokens: 16,384 per text or per contextual document for the 9b model (32,768 for the 0.6b),
> # counting one extra token per chunk in contextual mode, and 120,000 per request. The two character budgets below are
> # heuristics tuned for this corpus, which runs about 3.6 characters per token overall and close to 3 in its link-heavy
> # release notes. A corpus near 2 characters per token (CJK text, minified code, long link lists) can exceed either cap
> # with these values; lower both, or count tokens with the published tokenizer.
> # A document is one file, unless the file is longer than this or has more than REQUEST_CHUNKS chunks, in which case its
> # sections are split into consecutive documents.
> DOC_CHARS = 40_000
> # Per request: 512 texts or documents, 16,000 chunks in a contextual request (the documented limit), and this many
> # characters of text as the budget for the 120,000-token cap.
> REQUEST_TEXTS = 512
> REQUEST_CHUNKS = 16_000
> REQUEST_CHARS = 240_000
> INDEX_DIR = Path("index")
> # How deep eval looks for the answer. A question whose answer ranks below this counts as not found.
> EVAL_DEPTH = 100
>
>
> # --- chunk
> @dataclass
> class Chunk:
>     doc: str  # document name: the file path, with #2, #3 for the later parts of a split file
>     heading: str
>     text: str
>
>
> def track_fence(line: str, fence: str) -> str:
>     """The code fence that is open after this line, or "" outside code. A fence opens with three or more backticks or
>     tildes and closes only on a line of at least as many of the same character, so a shorter fence nested inside a
>     longer one (a three-backtick Python block inside a four-backtick Markdown block) does not close the outer block."""
>     match = re.match(r" {0,3}(`{3,}|~{3,})(.*)", line)
>     if not match:
>         return fence
>     marker, rest = match.groups()
>     if not fence:
>         return marker
>     if marker[0] == fence[0] and len(marker) >= len(fence) and not rest.strip():
>         return ""
>     return fence
>
>
> def split_long(section: str) -> list[str]:
>     """Split one section at blank lines outside code fences into pieces of about CHUNK_CHARS. A paragraph or code block
>     longer than that on its own is split at line breaks, and a single line longer than that is cut into CHUNK_CHARS pieces."""
>     paragraphs, current, fence = [], "", ""
>     for line in section.splitlines(keepends=True):
>         fence = track_fence(line, fence)
>         if not line.strip() and not fence and current:
>             paragraphs.append(current.strip("\n"))
>             current = ""
>         else:
>             current += line
>     if current.strip():
>         paragraphs.append(current.strip("\n"))
>     pieces = []
>     for paragraph in paragraphs:
>         if len(paragraph) <= CHUNK_CHARS:
>             pieces.append(paragraph)
>         else:
>             lines = [line[i : i + CHUNK_CHARS] for line in paragraph.splitlines() for i in range(0, len(line) or 1, CHUNK_CHARS)]
>             pieces.extend(_group(lines, "\n"))
>     return _group(pieces, "\n\n")
>
>
> def _group(parts: list[str], sep: str) -> list[str]:
>     pieces, current = [], ""
>     for part in parts:
>         if current and len(current) + len(part) > CHUNK_CHARS:
>             pieces.append(current.strip())
>             current = ""
>         current += part + sep
>     if current.strip():
>         pieces.append(current.strip())
>     return pieces
>
>
> def chunk_markdown(text: str) -> list[tuple[str, str]]:
>     """Return (heading, chunk text) pairs. Each heading starts a chunk and stays in it; headings inside code fences do not count."""
>     sections: list[str] = []
>     fence = ""
>     for line in text.splitlines(keepends=True):
>         fence = track_fence(line, fence)
>         if sections and (fence or not re.match(r"#{1,6} ", line)):
>             sections[-1] += line
>         else:
>             sections.append(line)
>     chunks = []
>     heading = ""
>     for section in sections:
>         if not section.strip():
>             continue
>         if section.startswith("#"):
>             heading = section.splitlines()[0].lstrip("#").strip()
>         for piece in split_long(section):
>             chunks.append((heading, piece))
>     return chunks
>
>
> def load_folder(root: str) -> list[list[Chunk]]:
>     """One document per Markdown file, in path order. A file longer than DOC_CHARS or with more than REQUEST_CHUNKS
>     chunks becomes several documents, so that a document stays within the character budget for the per-document cap."""
>     documents = []
>     for path in sorted(Path(root).rglob("*.md")):
>         name = str(path.relative_to(root))
>         parts: list[list[Chunk]] = [[]]
>         for heading, text in chunk_markdown(path.read_text(encoding="utf-8")):
>             if parts[-1] and (len(parts[-1]) == REQUEST_CHUNKS or sum(len(c.text) for c in parts[-1]) + len(text) > DOC_CHARS):
>                 parts.append([])
>             parts[-1].append(Chunk(name, heading or path.name, text))
>         for i, part in enumerate(parts):
>             if part:
>                 if i:
>                     for chunk in part:
>                         chunk.doc = f"{name}#{i + 1}"
>                 documents.append(part)
>     return documents
> ```

### One endpoint, two input shapes

`embed` builds every request the same way: `model`, `input_type`, `input`, `dimensions`, and `encoding_format`. The shape of `input` selects the mode. A flat list of strings is dense input and the response has one `embedding` per item in `data`. A list of lists is contextual input, one inner list per document, and the response nests one `data` list per document with one `embedding` per chunk, even for a one-chunk document. The tool sends questions as flat strings with `input_type: "query"`, because a question has no surrounding document; that is the recipe's choice, not an API rule. Responses are sorted by `index` at both levels before the vectors are stacked, so row `i` of the matrix is chunk `i` whatever order the items arrive in, and a response whose indices are not exactly 0 to n-1 is an error rather than a shifted matrix.

Vectors arrive as `base64_int8`, signed 8-bit integers, which is the default encoding and the only one besides packed binary; asking for `float` is a 400. `decode` turns the string into a numpy int8 vector and checks that its length is the `dimensions` the request asked for. The tool keeps the vectors as int8 from here on.

`batches` splits the inputs into requests within three limits: 512 texts or documents per request and 16,000 chunks per contextual request, both from the API reference, and 120,000 tokens per request, checked live. The token cap is enforced as a budget of 240,000 characters, `REQUEST_CHARS`, sized for this corpus like `DOC_CHARS`; if the API still returns a 400 for a request over the token cap, lower both (see Troubleshooting). An item that is over a limit on its own raises a `ValueError` naming the item and the limits instead of going out in a request the API will reject. `post` retries a 429 or a 5xx with backoff and stops on any other error with the API's own message, so a rejected parameter is readable. `record` adds up requests and tokens per model, and cost when the response reports it.

> [accordion] docsearch.py: embed
>
> ```python theme={null}
> # --- embed
> def batches(items: list, size_of, limits: tuple[int, ...]):
>     """Yield consecutive slices whose summed sizes stay within limits. size_of returns one count per limit for an item.
>     An item that is over a limit on its own cannot be sent, so that is an error here rather than a rejected request."""
>     start, totals = 0, [0] * len(limits)
>     for i, item in enumerate(items):
>         sizes = size_of(item)
>         if any(s > lim for s, lim in zip(sizes, limits, strict=True)):
>             raise ValueError(f"item {i} alone measures {sizes}; the request limits are {limits}")
>         if i > start and any(t + s > lim for t, s, lim in zip(totals, sizes, limits)):
>             yield items[start:i]
>             start, totals = i, [0] * len(limits)
>         totals = [t + s for t, s in zip(totals, sizes)]
>     if start < len(items):
>         yield items[start:]
>
>
> def post(client: httpx.Client, body: dict) -> dict:
>     """One POST to /v1/embeddings. Retries 429 and 5xx with backoff; any other error stops with the API's message."""
>     for attempt in range(5):
>         response = client.post(API_URL, json=body)
>         if response.status_code == 429 or response.status_code >= 500:
>             time.sleep(2**attempt)
>             continue
>         if response.status_code != 200:
>             raise RuntimeError(f"HTTP {response.status_code} from the Embeddings API: {response.text}")
>         return response.json()
>     raise RuntimeError(f"HTTP {response.status_code} from the Embeddings API after 5 attempts: {response.text}")
>
>
> def decode(b64: str, dimensions: int) -> np.ndarray:
>     """A base64_int8 embedding as a signed int8 vector, checked against the dimensions the request asked for."""
>     vector = np.frombuffer(base64.b64decode(b64), dtype=np.int8)
>     if len(vector) != dimensions:
>         raise RuntimeError(f"the API returned a {len(vector)}-dimensional vector for a {dimensions}-dimensional request")
>     return vector
>
>
> def embed(client: httpx.Client, model: str, input_type: str, inputs: list, dimensions: int, usage: dict) -> np.ndarray:
>     """Int8 vectors, one row per text or per chunk, in input order. A flat list of strings is dense input: each text
>     is embedded alone. A list of lists is contextual input: each inner list is one document, and every chunk vector
>     reflects the rest of its document. input_type is "query" for questions and "document" for everything indexed."""
>     contextual = bool(inputs) and isinstance(inputs[0], list)
>     if contextual:
>         size_of = lambda doc: (1, len(doc), sum(len(c) for c in doc))
>         limits = (REQUEST_TEXTS, REQUEST_CHUNKS, REQUEST_CHARS)
>     else:
>         size_of = lambda text: (1, len(text))
>         limits = (REQUEST_TEXTS, REQUEST_CHARS)
>     rows = []
>     for batch in batches(inputs, size_of, limits):
>         body = {"model": model, "input_type": input_type, "input": batch, "dimensions": dimensions, "encoding_format": "base64_int8"}
>         response = post(client, body)
>         record(usage, model, response)
>         for item in in_order(response["data"]):
>             if contextual:
>                 rows.extend(decode(c["embedding"], dimensions) for c in in_order(item["data"]))
>             else:
>                 rows.append(decode(item["embedding"], dimensions))
>     return np.stack(rows)
>
>
> def in_order(items: list[dict]) -> list[dict]:
>     """Response items sorted by index, after checking that the indices are exactly 0..n-1, so a response with a missing
>     or repeated item cannot shift every row after it."""
>     ordered = sorted(items, key=lambda d: d["index"])
>     if [d["index"] for d in ordered] != list(range(len(ordered))):
>         raise RuntimeError(f"the API returned indices {[d['index'] for d in ordered]}; expected 0 to {len(ordered) - 1}")
>     return ordered
>
>
> def record(usage: dict, model: str, response: dict) -> None:
>     """Add up requests, tokens, and cost per model. Cost is summed only when the response reports it."""
>     entry = usage.setdefault(model, {"requests": 0, "tokens": 0, "cost_usd": None})
>     entry["requests"] += 1
>     entry["tokens"] += response.get("usage", {}).get("total_tokens", 0)
>     cost = response.get("usage", {}).get("cost", {}).get("total_cost")
>     if cost is not None:
>         entry["cost_usd"] = (entry["cost_usd"] or 0.0) + cost
> ```

### Build, save, and load the index

`build_index` chunks the folder and embeds it once per distinct index in `CONFIGS`: every chunk alone at 2048 dimensions, every document's chunks together at 2048, and the same at 1024. Three files go to `index/`: `vectors.npz` with one int8 matrix per index, `chunks.json` with the document, heading, and text of each chunk, and `models.json` with the model id and the mode and dimensions of each matrix. Int8 keeps 2048 dimensions in 2 KB per chunk and 1024 in 1 KB; the three matrices for 2,683 chunks total 13.7 MB, where float32 would take 55 MB.

An `Index` is a matrix and the float32 norm of each row. `rank` orders rows by cosine: an integer dot product divided by the row norms, with the query's norm left out because it is the same for every row and does not change the order. int8 times int8 summed over 2048 dimensions stays far inside int32, and the division runs in float32. `load_index` refuses an index built with another model, one missing a matrix a configuration needs, or one whose saved matrices do not have one row per chunk and the recorded dtype and dimensions. `embed_queries` embeds the questions once per configuration, with that configuration's query model and dimensions; the small model produces 512 or 1024 dimensions and never 2048, so `rank` also checks the query against the matrix, and a 1024-dimensional question against the 2048 index is an error rather than a wrong answer.

> [accordion] docsearch.py: index
>
> ```python theme={null}
> # --- index
> @dataclass
> class Index:
>     """One int8 matrix (a row per chunk) and the float32 norm of each row, so cosine similarity is an integer dot
>     product divided by two norms. Storing int8 keeps 2,048 dimensions in 2 KB per chunk and 1,024 in 1 KB."""
>
>     matrix: np.ndarray
>     norms: np.ndarray
>
>     @classmethod
>     def of(cls, matrix: np.ndarray) -> "Index":
>         return cls(matrix, np.linalg.norm(matrix.astype(np.float32), axis=1))
>
>
> def build_index(client: httpx.Client, root: str, out: Path = INDEX_DIR) -> dict:
>     """Embed every chunk under root once per distinct (mode, dimensions) in CONFIGS, with INDEX_MODEL, and write the
>     int8 matrices, the chunk metadata, and a description of each matrix to out."""
>     documents = load_folder(root)
>     chunks = [chunk for doc in documents for chunk in doc]
>     usage: dict = {}
>     started = time.time()
>     matrices: dict[str, np.ndarray] = {}
>     for config in CONFIGS:
>         if config.index_key in matrices:
>             continue
>         if config.mode == "dense":
>             inputs: list = [c.text for c in chunks]
>         else:
>             inputs = [[c.text for c in doc] for doc in documents]
>         matrix = embed(client, INDEX_MODEL, "document", inputs, config.dimensions, usage)
>         if len(matrix) != len(chunks):
>             raise RuntimeError(f"{config.index_key}: embedding count {len(matrix)} does not match chunk count {len(chunks)}")
>         matrices[config.index_key] = matrix
>     out.mkdir(exist_ok=True)
>     np.savez(out / "vectors.npz", **matrices)
>     (out / "chunks.json").write_text(json.dumps([asdict(c) for c in chunks]))
>     described = {c.index_key: {"mode": c.mode, "dimensions": c.dimensions} for c in CONFIGS}
>     (out / "models.json").write_text(json.dumps({"model": INDEX_MODEL, "indexes": described}))
>     return {"documents": len(documents), "chunks": len(chunks), "seconds": time.time() - started, "usage": usage}
>
>
> def load_index(path: Path = INDEX_DIR) -> tuple[list[Chunk], dict[str, Index]]:
>     """Read the index back. Refuses an index built with another model, one that lacks a matrix a config needs, or one
>     whose saved vectors do not have one row per chunk and the dimensions recorded when it was built."""
>     models = json.loads((path / "models.json").read_text())
>     if models["model"] != INDEX_MODEL:
>         raise RuntimeError(f"index was built with {models['model']}; run index again")
>     missing = sorted({c.index_key for c in CONFIGS} - set(models["indexes"]))
>     if missing:
>         raise RuntimeError(f"index has no vectors for {', '.join(missing)}; run index again")
>     chunks = [Chunk(**c) for c in json.loads((path / "chunks.json").read_text())]
>     vectors = np.load(path / "vectors.npz")
>     indexes = {}
>     for key, described in models["indexes"].items():
>         matrix = vectors[key]
>         if matrix.dtype != np.int8 or matrix.ndim != 2 or matrix.shape[1] != described["dimensions"]:
>             raise RuntimeError(f"{key} vectors are {matrix.dtype} with shape {matrix.shape}, models.json says int8 with {described['dimensions']} dimensions; run index again")
>         if matrix.shape[0] != len(chunks):
>             raise RuntimeError(f"{key} has {matrix.shape[0]} vectors for {len(chunks)} chunks; run index again")
>         indexes[key] = Index.of(matrix)
>     return chunks, indexes
>
>
> def embed_queries(client: httpx.Client, questions: list[str]) -> dict[str, np.ndarray]:
>     """One query matrix per config, embedded with the config's query model at the config's dimensions. Questions go
>     as flat strings with input_type "query": a question has no surrounding document."""
>     usage: dict = {}
>     queries = {}
>     for config in CONFIGS:
>         queries[config.name] = embed(client, config.query_model, "query", questions, config.dimensions, usage)
>     return queries
>
>
> def rank(index: Index, query: np.ndarray) -> np.ndarray:
>     """Row indices from most to least similar, ranked by cosine: the dot product over the query's norm times the row's
>     norm, with the query's norm left out because it is the same for every row. The dot product runs in int32 (int8
>     times int8 summed over 2,048 dimensions cannot overflow it) and the row norms divide it in float32."""
>     if query.shape[-1] != index.matrix.shape[1]:
>         raise RuntimeError(f"query vector has {query.shape[-1]} dimensions but the index has {index.matrix.shape[1]}; run index again")
>     scores = (index.matrix @ query.astype(np.int32)) / index.norms
>     return np.argsort(-scores, kind="stable")
> ```

### Ask a question

`ask` loads the index, embeds the question for every configuration, and prints the top `k` chunks for the configurations you name with `--configs` (default: `dense-2048` and `context-2048`), with the file, the heading, and the first 160 characters of the chunk, so you can see what each would hand to a reader or a RAG prompt.

> [accordion] docsearch.py: ask
>
> ```python theme={null}
> # --- ask
> def ask(client: httpx.Client, question: str, k: int, names: list[str]) -> None:
>     chunks, indexes = load_index()
>     queries = embed_queries(client, [question])
>     for config in [c for c in CONFIGS if c.name in names]:
>         print(f"\n{config.name}: {config.mode} index at {config.dimensions} dimensions, question embedded by {config.query_model}")
>         for place, row in enumerate(rank(indexes[config.index_key], queries[config.name][0])[:k], start=1):
>             chunk = chunks[row]
>             preview = " ".join(chunk.text.split())[:160].rstrip()
>             print(f"  {place}. {chunk.doc} > {chunk.heading}\n     {preview}")
> ```

### Score the configurations on labeled questions

A label is a question, the file that answers it, and a phrase the answering chunk contains, or a list of phrases when the answer appears in more than one form. `hit` says a chunk answers the question when it comes from that file (any part of a split file counts, and only the `#2` part suffix is stripped, so a file named `C# guide.md` still matches) and contains any of the phrases, case-insensitively. `check_labels` runs first and stops on a label with no phrase, a repeated question, or a label that matches no chunk in the index, so a typo in a phrase shows up as an error rather than as a zero. `evaluate` records the rank of the first hit in the top `EVAL_DEPTH` (100) for each question and configuration, or 0 when there is none. `summarize` turns the ranks into recall\@1, recall\@3, recall\@5, and MRR\@100, which is MRR with every answer below rank 100 counted as zero. `print_eval` shows the summary and then a per-question table, because the table is where you learn which sections each configuration handles well.

> [accordion] docsearch.py: eval and output
>
> ```python theme={null}
> # --- eval
> def phrases(label: dict) -> list[str]:
>     """The `contains` field is one phrase or a list of phrases; a chunk that has any of them is accepted."""
>     value = label["contains"]
>     return [value] if isinstance(value, str) else list(value)
>
>
> def hit(chunk: Chunk, label: dict) -> bool:
>     """A chunk answers a question when it comes from the labeled file and contains one of the labeled phrases. Only the
>     #2, #3 part suffix of a split file is removed before comparing, so a file named `C# guide.md` keeps its name."""
>     text = chunk.text.lower()
>     return re.sub(r"#\d+$", "", chunk.doc) == label["file"] and any(p.lower() in text for p in phrases(label) if p.strip())
>
>
> def check_labels(chunks: list[Chunk], labels: list[dict]) -> None:
>     """Stop before spending any tokens on a label file that cannot be scored: an empty phrase would accept every chunk
>     of a file, a repeated question would count twice, and a label that matches no chunk is a typo in the file or phrase,
>     or a corpus that changed since the label was written."""
>     if any(not phrases(label) or not all(p.strip() for p in phrases(label)) for label in labels):
>         sys.exit("every label needs at least one non-empty phrase in 'contains'")
>     questions = [label["question"] for label in labels]
>     if len(set(questions)) != len(questions):
>         sys.exit("duplicate question in the label file: " + next(q for q in questions if questions.count(q) > 1))
>     unmatched = [label for label in labels if not any(hit(chunk, label) for chunk in chunks)]
>     if unmatched:
>         lines = [f"  {label['file']} containing {label['contains']!r}: {label['question']}" for label in unmatched]
>         sys.exit(f"{len(unmatched)} label(s) match no chunk in the index. Fix the label or rebuild the index.\n" + "\n".join(lines))
>
>
> def evaluate(chunks: list[Chunk], indexes: dict[str, Index], queries: dict, labels: list[dict]) -> dict[str, list[int]]:
>     """For each config, the rank of the first chunk that answers each question (0 when none in the top EVAL_DEPTH)."""
>     ranks: dict[str, list[int]] = {}
>     for config in CONFIGS:
>         ranks[config.name] = []
>         for label, query in zip(labels, queries[config.name], strict=True):
>             order = rank(indexes[config.index_key], query)[:EVAL_DEPTH]
>             found = next((i + 1 for i, row in enumerate(order) if hit(chunks[row], label)), 0)
>             ranks[config.name].append(found)
>     return ranks
>
>
> def summarize(ranks: list[int]) -> dict[str, float]:
>     """recall@1, 3, 5 and MRR over the top EVAL_DEPTH, so the metric is MRR@100 and an answer below that counts as zero."""
>     found = [r for r in ranks if r]
>     result = {f"recall@{k}": sum(1 for r in found if r <= k) / len(ranks) for k in (1, 3, 5)}
>     result[f"MRR@{EVAL_DEPTH}"] = sum(1 / r for r in found) / len(ranks)
>     return result
>
>
> # --- output
> def print_eval(labels: list[dict], ranks: dict[str, list[int]]) -> None:
>     names = [c.name for c in CONFIGS]
>     width = max(len(n) for n in names)
>     metrics = {name: summarize(ranks[name]) for name in names}
>     print(f"{len(labels)} questions\n")
>     print(f"{'configuration':<{width}}  " + "  ".join(f"{k:>8}" for k in metrics[names[0]]))
>     for name in names:
>         print(f"{name:<{width}}  " + "  ".join(f"{v:>8.3f}" for v in metrics[name].values()))
>     print(f"\nRank of the first chunk that answers each question (1 is best, - means not in the top {EVAL_DEPTH})")
>     print(f"{'question':<62}  " + "  ".join(f"{n:>{len(n)}}" for n in names))
>     for i, label in enumerate(labels):
>         print(f"{label['question'][:62]:<62}  " + "  ".join(f"{ranks[n][i] or '-':>{len(n)}}" for n in names))
>
>
> def main(argv: list[str] | None = None) -> None:
>     parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
>     sub = parser.add_subparsers(dest="command", required=True)
>     sub.add_parser("index").add_argument("folder")
>     p_ask = sub.add_parser("ask")
>     p_ask.add_argument("question")
>     p_ask.add_argument("-k", type=int, default=3)
>     p_ask.add_argument("--configs", nargs="+", choices=[c.name for c in CONFIGS], default=["dense-2048", "context-2048"])
>     sub.add_parser("eval").add_argument("questions")
>     args = parser.parse_args(argv)
>     key = os.environ.get("PERPLEXITY_API_KEY")
>     if not key:
>         sys.exit("export PERPLEXITY_API_KEY before running this script")
>     client = httpx.Client(headers={"Authorization": f"Bearer {key}"}, timeout=120)
>
>     if args.command == "index":
>         report = build_index(client, args.folder)
>         print(f"{report['documents']} documents, {report['chunks']} chunks, indexed in {report['seconds']:.0f} s")
>         for model, entry in report["usage"].items():
>             cost = f"${entry['cost_usd']:.4f}" if entry["cost_usd"] is not None else "cost not reported by the API"
>             print(f"  {model}: {entry['requests']} requests, {entry['tokens']:,} tokens, {cost}")
>     elif args.command == "ask":
>         ask(client, args.question, args.k, args.configs)
>     else:
>         labels = [json.loads(line) for line in Path(args.questions).read_text().splitlines() if line.strip()]
>         chunks, indexes = load_index()
>         check_labels(chunks, labels)
>         queries = embed_queries(client, [label["question"] for label in labels])
>         print_eval(labels, evaluate(chunks, indexes, queries, labels))
>
>
> if __name__ == "__main__":
>     main()
> ```

## Index the folder

```bash theme={null}
python docsearch.py index fastapi-docs
```

Observed on November 4, 2026 on Linux with Python 3.12, with the files extracted from this page:

```text theme={null}
168 documents, 2683 chunks, indexed in 39 s
  pplx-embed-v2-context-9b: 30 requests, 1,815,960 tokens, $0.2179
```

The chunker produces 168 documents and 2,683 chunks from the 144 files: seven files are longer than 40,000 characters once their code is inlined and split into parts, `release-notes.md` into twelve. Each pass takes ten requests under the 240,000-character budget, thirty in all. The API counted 1,815,960 tokens for the three passes, about 605,000 per pass, and reported \$0.2179 in `usage.cost`, which the tool sums. `index/` is 16 MB: 13.7 MB of int8 vectors in `vectors.npz` and 2.5 MB of chunk text in `chunks.json`.

## Ask it questions

Each `ask` embeds the question once per configuration and prints the rankings you ask for. Two more from the recorded run, chosen because the modes disagree:

````text theme={null}
$ python docsearch.py ask "How do I open a websocket connection?"

dense-2048: dense index at 2048 dimensions, question embedded by pplx-embed-v2-context-9b
  1. advanced/websockets.md > WebSockets client
     ## WebSockets client
  2. advanced/websockets.md > In production
     ### In production In your production system, you probably have a frontend created with a modern framework like React, Vue.js or Angular. And to communicate usin
  3. advanced/websockets.md > Create a `websocket`
     ## Create a `websocket` In your **FastAPI** application, create a `websocket`:

context-2048: context index at 2048 dimensions, question embedded by pplx-embed-v2-context-9b
  1. advanced/websockets.md > Create a `websocket`
     ## Create a `websocket` In your **FastAPI** application, create a `websocket`:
  2. advanced/websockets.md > Try the WebSockets with dependencies
     ### Try the WebSockets with dependencies If your file is named `main.py`, run your application with: <div class="termy"> ```console $ fastapi dev main.py <span
  3. advanced/websockets.md > Install `WebSockets`
     ## Install `WebSockets` Make sure you create a [virtual environment](../virtual-environments.md){.internal-link target=_blank}, activate it, and install `websoc
````

Both configurations stay on `advanced/websockets.md`, and neither puts the code first. Dense mode's first result is the chunk `## WebSockets client`, a heading with no body of its own (the content is in its subsections), followed by the "In production" prose and then the heading line of "Create a `websocket`". Contextual mode puts that heading line first, then the "Try the WebSockets with dependencies" and "Install `WebSockets`" sections. The label is the code, `@app.websocket`, which appears in ten chunks of the page, all of them inlined examples that are mostly an HTML page inside a Python string with the decorator near the end. The first of those is rank 6 for `context-2048` and rank 11 for `dense-2048`, so by the label both configurations miss the top 5 on this question, and a reader who opened the contextual result would be one heading above the code they wanted.

````text theme={null}
$ python docsearch.py ask "Where do I put code that runs once when the server starts?"

dense-2048: dense index at 2048 dimensions, question embedded by pplx-embed-v2-context-9b
  1. deployment/manually.md > Run a Server Manually
     # Run a Server Manually
  2. deployment/concepts.md > Separate Program
     ### Separate Program To achieve this, you will normally have a **separate program** that would make sure your application is run on startup. And in many cases,
  3. deployment/concepts.md > Run Automatically on Startup
     ### Run Automatically on Startup In general, you will probably want the server program (e.g. Uvicorn) to be started automatically on server startup, and without

context-2048: context index at 2048 dimensions, question embedded by pplx-embed-v2-context-9b
  1. advanced/events.md > Lifespan Events
     # Lifespan Events You can define logic (code) that should be executed before the application **starts up**. This means that this code will be executed **once**,
  2. advanced/events.md > `startup` event
     ### `startup` event To add a function that should be run before the application starts, declare it with the event `"startup"`: ```Python hl_lines="8" from fasta
  3. deployment/manually.md > Use the `fastapi run` Command
     ## Use the `fastapi run` Command In short, use `fastapi run` to serve your FastAPI application: <div class="termy">
````

The first three dense hits are about the server process and boot behavior: `deployment/manually.md` and two sections of `deployment/concepts.md`; the first chunk that shows how to register startup code is the seventh. Contextual mode puts `advanced/events.md` first: the page's opening section, which describes code that runs once before the application starts receiving requests but does not yet name the parameter, and then the `startup` event section with `@app.on_event("startup")`, which the label accepts, at rank 2. The `lifespan` parameter, the current way to do this, is in the "Lifespan" section further down the same page.

## Score the configurations

`questions.jsonl` holds 30 questions a FastAPI user might ask, each labeled with the file that answers it and a phrase the answering chunk contains, or a list of phrases where the docs state the answer in more than one way. The questions are written the way people ask them ("How do I turn off the automatic documentation pages?") rather than with the words of the heading ("Docs URLs"), and the phrases are the API call or the fact a reader is looking for (`docs_url`, `convert_underscores`, `status_code=201`). Every label was checked chunk by chunk against the index: each chunk a label accepts was read against its question, and phrases were tightened where they accepted a chunk that only mentions the topic and widened where they rejected a chunk that answers it.

> [accordion] questions.jsonl
>
> ```json questions.jsonl theme={null}
> {"question": "How do I run my tests?", "file": "tutorial/testing.md", "contains": "$ pytest"}
> {"question": "How do I make a path parameter accept values with slashes in them?", "file": "tutorial/path-params.md", "contains": ":path"}
> {"question": "How do I run a function after returning a response, like sending an email?", "file": "tutorial/background-tasks.md", "contains": ".add_task("}
> {"question": "How do I run the development server with auto reload?", "file": "fastapi-cli.md", "contains": "fastapi dev"}
> {"question": "What status code is returned when request validation fails?", "file": "tutorial/handling-errors.md", "contains": "422"}
> {"question": "How do I stop underscores in header names from being converted to hyphens?", "file": "tutorial/header-params.md", "contains": "convert_underscores"}
> {"question": "How do I leave default values out of the response body?", "file": "tutorial/response-model.md", "contains": ["response_model_exclude_unset", "response_model_exclude_defaults"]}
> {"question": "How do I apply one dependency to every route in a router?", "file": "tutorial/bigger-applications.md", "contains": ["we can add it to the `APIRouter`", "dependencies=[Depends(get_token_header)]", "all the *path operations* in the router", "`dependencies` in the `APIRouter`"]}
> {"question": "How do I mark an endpoint as deprecated in the docs?", "file": "tutorial/path-operation-configuration.md", "contains": "deprecated"}
> {"question": "How do I turn off the automatic documentation pages?", "file": "tutorial/metadata.md", "contains": "docs_url"}
> {"question": "How do I serve a folder of images at a URL path?", "file": "tutorial/static-files.md", "contains": "StaticFiles(directory"}
> {"question": "How do I allow requests from a frontend running on another origin?", "file": "tutorial/cors.md", "contains": "allow_origins"}
> {"question": "How do I log in with a username and password from the docs UI?", "file": "tutorial/security/simple-oauth2.md", "contains": "\"Authorize\" button"}
> {"question": "How do I run FastAPI behind a proxy that strips a path prefix?", "file": "advanced/behind-a-proxy.md", "contains": "--root-path"}
> {"question": "Where do I put code that runs once when the server starts?", "file": "advanced/events.md", "contains": ["`lifespan` parameter", "@app.on_event(\"startup\")"]}
> {"question": "How do I convert a datetime to something JSON can store?", "file": "tutorial/encoder.md", "contains": "jsonable_encoder"}
> {"question": "How do I see the request body that failed validation?", "file": "tutorial/handling-errors.md", "contains": "exc.body"}
> {"question": "How do I upload several files at once?", "file": "tutorial/request-files.md", "contains": "[bytes]"}
> {"question": "How do I read settings from environment variables?", "file": "advanced/settings.md", "contains": "BaseSettings"}
> {"question": "How do I replace a dependency with a fake one in tests?", "file": "advanced/testing-dependencies.md", "contains": ["app.dependency_overrides[", "as a key the original dependency"]}
> {"question": "How do I set a cookie on the response?", "file": "advanced/response-cookies.md", "contains": "set cookies"}
> {"question": "How do I stream a large file back to the client?", "file": "advanced/custom-response.md", "contains": "StreamingResponse"}
> {"question": "How do I show example values for a request body in the docs?", "file": "tutorial/schema-extra-example.md", "contains": "examples=["}
> {"question": "How do I accept the same query parameter several times?", "file": "tutorial/query-params-str-validations.md", "contains": "list[str]"}
> {"question": "How do I return 201 when an item is created?", "file": "tutorial/response-status-code.md", "contains": "status_code=201"}
> {"question": "How do I group query parameters into a Pydantic model?", "file": "tutorial/query-param-models.md", "contains": ["FilterParams, Query()", "FilterParams = Query()"]}
> {"question": "How do I get the current user from a token in every route?", "file": "tutorial/security/get-current-user.md", "contains": "Depends(get_current_user)"}
> {"question": "How do I return a file from disk as the response?", "file": "advanced/custom-response.md", "contains": "FileResponse"}
> {"question": "How do I validate that a number path parameter is at least 1?", "file": "tutorial/path-params-numeric-validations.md", "contains": "ge=1"}
> {"question": "How do I open a websocket connection?", "file": "advanced/websockets.md", "contains": "@app.websocket"}
> ```

```bash theme={null}
python docsearch.py eval questions.jsonl
```

```text theme={null}
30 questions

configuration       recall@1  recall@3  recall@5   MRR@100
dense-2048             0.267     0.700     0.767     0.493
context-2048           0.567     0.867     0.933     0.724
context-1024           0.567     0.833     0.867     0.715
context-1024-small     0.533     0.767     0.800     0.679
```

> [accordion] Rank of the first answering chunk for each question and configuration
>
> ```text theme={null}
> Rank of the first chunk that answers each question (1 is best, - means not in the top 100)
> question                                                        dense-2048  context-2048  context-1024  context-1024-small
> How do I run my tests?                                                   2             2             2                   2
> How do I make a path parameter accept values with slashes in t           6             1             1                   1
> How do I run a function after returning a response, like sendi           3             2             2                   1
> How do I run the development server with auto reload?                    1             1             1                   1
> What status code is returned when request validation fails?              2             5             6                   6
> How do I stop underscores in header names from being converted           1             1             1                   1
> How do I leave default values out of the response body?                  1             1             1                   1
> How do I apply one dependency to every route in a router?                1             7             7                   7
> How do I mark an endpoint as deprecated in the docs?                     2             2             2                   2
> How do I turn off the automatic documentation pages?                     3             5             9                  10
> How do I serve a folder of images at a URL path?                        10             2             2                   2
> How do I allow requests from a frontend running on another ori           4             1             1                   2
> How do I log in with a username and password from the docs UI?           1             1             1                   1
> How do I run FastAPI behind a proxy that strips a path prefix?           6             1             1                   1
> Where do I put code that runs once when the server starts?               7             2             2                   6
> How do I convert a datetime to something JSON can store?                 2             1             1                   1
> How do I see the request body that failed validation?                    2             1             1                   1
> How do I upload several files at once?                                   1             1             1                   1
> How do I read settings from environment variables?                       9             3             3                   7
> How do I replace a dependency with a fake one in tests?                  5             1             1                   1
> How do I set a cookie on the response?                                   2             1             1                   1
> How do I stream a large file back to the client?                         1             1             1                   1
> How do I show example values for a request body in the docs?             3             3             3                   3
> How do I accept the same query parameter several times?                  3             1             1                   1
> How do I return 201 when an item is created?                             3             3             4                   5
> How do I group query parameters into a Pydantic model?                   2             2             2                   2
> How do I get the current user from a token in every route?              14             1             1                   2
> How do I return a file from disk as the response?                        1             1             1                   1
> How do I validate that a number path parameter is at least 1?            3             1             1                   1
> How do I open a websocket connection?                                   11             6             8                   8
> ```

Two runs of `eval` on the same index produced byte-identical output, summary and table.

## Read the numbers

`context-2048` beats `dense-2048` on every summary number: recall\@1 0.567 against 0.267, recall\@3 0.867 against 0.700, recall\@5 0.933 against 0.767, MRR\@100 0.724 against 0.493. Per question, the contextual index ranks the answer higher on 15 of the 30, lower on 3, and the same on 12, ten of those ties at rank 1 or 2 for both.

The largest moves: "How do I get the current user from a token in every route?" goes from 14 to 1; the label accepts 25 chunks of `tutorial/security/get-current-user.md`, the same complete example repeated once per Python version tab. "How do I serve a folder of images at a URL path?" goes from 10 to 2; the one chunk the label accepts, with `StaticFiles(directory="static")`, says "mount" and "static", never "images" or "folder". "How do I run FastAPI behind a proxy that strips a path prefix?" goes from 6 to 1; the label, `--root-path`, accepts five chunks of `advanced/behind-a-proxy.md`. The path-convertor question (6 to 1) and the startup question (7 to 2) are the two `ask` examples above, and the websocket question (11 to 6) is the one where both configurations miss the top 5.

Three questions moved the other way. "How do I apply one dependency to every route in a router?" goes from 1 to 7: the contextual index puts the Global Dependencies pages ahead of `tutorial/bigger-applications.md`, and they use the same `dependencies=` argument on the application instead of the router. The recorded `ask` below shows it. "What status code is returned when request validation fails?" goes from 2 to 5: the only chunk of `tutorial/handling-errors.md` that contains `422` is the example that overrides the validation handler, so both ranks measure finding that example rather than the fact itself. "How do I turn off the automatic documentation pages?" goes from 3 to 5: the answer, `docs_url=None`, is one section of `tutorial/metadata.md`, a page otherwise about titles, versions, and tags.

````text theme={null}
$ python docsearch.py ask "How do I apply one dependency to every route in a router?" -k 4 --configs context-2048 context-1024-small

context-2048: context index at 2048 dimensions, question embedded by pplx-embed-v2-context-9b
  1. tutorial/dependencies/global-dependencies.md > Global Dependencies
     # Global Dependencies For some types of applications you might want to add dependencies to the whole application. Similar to the way you can [add `dependencies`
  2. tutorial/dependencies/dependencies-in-path-operation-decorators.md > Global Dependencies
     ## Global Dependencies Next we will see how to add dependencies to the whole `FastAPI` application, so that they apply to each *path operation*.
  3. tutorial/dependencies/global-dependencies.md > Dependencies for groups of *path operations*
     ## Dependencies for groups of *path operations* Later, when reading about how to structure bigger applications ([Bigger Applications - Multiple Files](../../tut
  4. tutorial/dependencies/global-dependencies.md > Global Dependencies
     ```Python hl_lines="15" from fastapi import Depends, FastAPI, Header, HTTPException async def verify_token(x_token: str = Header()): if x_token != "fake-super-s

context-1024-small: context index at 1024 dimensions, question embedded by pplx-embed-v2-context-0.6b
  1. tutorial/dependencies/global-dependencies.md > Global Dependencies
     # Global Dependencies For some types of applications you might want to add dependencies to the whole application. Similar to the way you can [add `dependencies`
  2. tutorial/dependencies/dependencies-in-path-operation-decorators.md > Dependencies in path operation decorators
     # Dependencies in path operation decorators In some cases you don't really need the return value of a dependency inside your *path operation function*. Or the d
  3. tutorial/dependencies/dependencies-in-path-operation-decorators.md > Global Dependencies
     ## Global Dependencies Next we will see how to add dependencies to the whole `FastAPI` application, so that they apply to each *path operation*.
  4. tutorial/dependencies/dependencies-in-path-operation-decorators.md > Add `dependencies` to the *path operation decorator*
     ## Add `dependencies` to the *path operation decorator* The *path operation decorator* receives an optional argument `dependencies`. It should be a `list` of `D
````

Both columns come from the contextual index: the left one is the 2048-dimensional matrix queried by the 9b model, the right one the 1024-dimensional matrix queried by the 0.6b model, so this one question shows both levers at once. Both put the Global Dependencies page first, and neither shows `tutorial/bigger-applications.md` within four results; the label is rank 7 in both columns of the table. The large query model keeps three of its four results on `global-dependencies.md`, including the code with `dependencies=[Depends(verify_token)]`; the small one spreads across the two dependency pages. Both configurations miss here; the two columns differ only in which chunks fill the four places.

### Query a 9b index with the 0.6b model

The `context-1024-small` column builds nothing new. It reuses the contextual 1024 index and embeds the questions with `pplx-embed-v2-context-0.6b`, which produces 1024-dimensional vectors in the same space. You would run a search service this way when the index is built once and queried often: the large model embeds the corpus once, and the small model handles the stream of questions, where latency and per-query cost add up. The gap between `context-1024` and `context-1024-small` is the price of that choice on your questions. Here it is 0.533, 0.767, 0.800, and 0.679 against 0.567, 0.833, 0.867, and 0.715: the small query model does better on one question, worse on six, and the same on 23.

### 1024 or 2048 dimensions

The 9b model returns 1024 dimensions when you ask for them, and the 0.6b model returns 512 or 1024, so 1024 is the one size both models share, and that is why the cross-size column uses it. At 1024, storage halves: 1 KB per chunk instead of 2 KB, 2.7 MB instead of 5.5 MB for this corpus, and a dot product over half as many values. The gap between `context-2048` and `context-1024` is what the second kilobyte buys on your questions. Here it is 0.567, 0.867, 0.933, and 0.724 against 0.567, 0.833, 0.867, and 0.715: the 1024 index is never better, equal on 26 questions, and worse on four. A 512-dimensional index needs the 0.6b model as the index model, which this tool does not do.

### Cautions

Thirty questions show a direction, not a precise number: one question is 0.033 of recall\@1, so a gap of one or two questions on a line is within what one author's question set can produce, and you should write more before you trust it. A label is a substring test, not a judgment, so a label that reads well can accept a chunk that only mentions its phrase; the only cure is to read what it accepts, which the index files make easy. Some labels accept several chunks of their file because the same code is repeated once per Python version tab (`Depends(get_current_user)` matches 25 chunks), so for those questions the table measures finding the right page and code more than one exact section; 10 of the 30 labels pin a single chunk. And the result is specific to this corpus and this chunker: move the chunk boundaries, or run on docs where sections lean on the surrounding page, and the table will move.

The useful conclusion is the one the tool is built for: on your docs, with your chunker and your questions, you can see which mode, which dimension count, and which query model find the right section, before you commit to a vector database.

## Migrate from v1

The v1 models keep working. `/v1/embeddings` serves the v1 dense ids and both v2 ids; `/v1/contextualizedembeddings` serves only the two v1 contextual ids and returns a 400 for a v2 id, and a v1 contextual id on `/v1/embeddings` is a 400 too. That was checked for `pplx-embed-v1-4b` and `pplx-embed-context-v1-4b` while this page was written. Vectors are not interchangeable across versions: a v2 question scored against a v1 index produces rankings that mean nothing, and `load_index` refuses an index whose `models.json` names another model. To move an index to v2, run `index` again with the v2 model; there is no conversion. The v2 endpoint requires `input_type` on every request and takes contextual input on `/v1/embeddings` as nested lists, where v1 used a separate endpoint.

## Adapt it to your docs folder

* **Your corpus.** `python docsearch.py index path/to/docs` reads every `.md` under the folder. For other formats, change `load_folder` to produce documents (ordered lists of `Chunk`) however you like; everything after it is format-agnostic.
* **Your chunker.** `chunk_markdown` splits at headings because that is how documentation is written. If your production pipeline chunks by token count, use that here instead, because the result depends on where the boundaries fall. Keep each document's chunks in reading order; the contextual mode depends on it.
* **Your questions.** Write 30 to 50 the way your users ask them, then record the file and a phrase from the chunk that answers each. Include easy ones. A set made only of hard cases tells you about the hard cases.
* **Your configurations.** Each `Config` line is one column. Drop the ones you do not need to save an embedding pass, or add `Config("dense-1024", "dense", 1024, INDEX_MODEL)` to see dense mode at the smaller size. A configuration whose query model is `SMALL_MODEL` has to use a 1024-dimensional index.
* **Your vector database.** The numpy index is enough to pick a mode, a dimension count, and a chunker. Once you have, store the int8 vectors where your application already keeps its records; the [Embeddings quickstart](/docs/embeddings/quickstart#add-embeddings-to-an-existing-project) shows that pattern, and the chunk text in `chunks.json` is what you would hand to a RAG prompt.

## Troubleshooting

* `export PERPLEXITY_API_KEY before running this script` means the key is not exported in the shell that runs the script.
* `FileNotFoundError: index/models.json` means `ask` or `eval` ran before `index`, or in a different directory. `models.json` is the first file `load_index` reads.
* `HTTP 400 from the Embeddings API: ...` quotes the API's message. The ones this tool can provoke, in the API's words: `input_type is required for model pplx-embed-v2-context-9b; use query or document` if you edit a request and drop the field; `Output dimensions must be one of [1024, 2048], got 512.` for a `dimensions` value the model does not serve (the 0.6b model answers with `[512, 1024]`); and `input_ids length 16922 must be between 1 and max sequence length (16384)` for a contextual document over the 9b cap, in which case lower `DOC_CHARS`. The 0.6b model reports the same case as `Document 0 has too many tokens: got 34296, maximum is 32768.` Any other `encoding_format` gets `encoding format must be one of: base64_int8 base64_binary`.
* `Input total size exceeds maximum number of allowed tokens: got 129618, maximum is 120000.` means a request passed 120,000 tokens inside the 240,000-character budget. Two causes: text under two characters per token (CJK text, minified code, long link lists), or a contextual request whose documents hold thousands of short chunks, since each chunk costs one extra token. Lower `REQUEST_CHARS`; if the documents are the cause, lower `DOC_CHARS` too.
* `HTTP 429 ... after 5 attempts` means the rate limit held for five tries with backoff up to 16 seconds. Rerun `index`.
* `ValueError: item N alone measures ...; the request limits are ...` means one document is over a request budget by itself. The folder loader splits documents at `REQUEST_CHUNKS` and `DOC_CHARS`, so you see this only if you raised `DOC_CHARS` above `REQUEST_CHARS` or changed the loader.
* `RuntimeError: the API returned a N-dimensional vector for a M-dimensional request` means the service ignored `dimensions`. The 9b model serves 1024 or 2048 and the 0.6b model 512 or 1024.
* `RuntimeError: the API returned indices [...]; expected 0 to N` means a response skipped or repeated an item, and `RuntimeError: ... embedding count ... does not match chunk count` means a response had fewer vectors than inputs. Rerun `index`.
* `RuntimeError: ... run index again` means the saved index does not match the constants or itself: another model id, a missing matrix after you added a `Config`, or a dtype, dimension, or row-count mismatch between `vectors.npz`, `chunks.json`, and `models.json`. Rebuild it.
* `N label(s) match no chunk in the index` from `eval` lists each label whose file and phrase match nothing. Fix the phrase, or rebuild the index if the corpus changed.

## Prove it without an API key

The 22 tests run offline against a fake Embeddings API behind an httpx transport. The fake answers both request shapes with shuffled base64 int8 vectors and rejects malformed requests. Its wording for a missing `input_type`, an unsupported `dimensions` value, and an unsupported `encoding_format` is the live API's; its other checks (mixed flat and nested input, empty strings, `num_chunks` on text) are the recipe's own, not verified API behavior. The tests cover the chunker and the folder loader, every request limit and the shape of every request, retry and error reporting in `post`, response ordering and decoding, the checks in `load_index` and `rank`, including the 0.6b query against each index, and the scoring functions as the Score section describes them. The file is in the next section.

```bash theme={null}
python -m pytest -q test_docsearch.py
```

```text theme={null}
......................                                                   [100%]
22 passed in 0.16s
```

## Copy the complete files

`requirements.txt` is in Set up and `questions.jsonl` in Score the configurations. The other two files are below, each in two parts so the blocks stay short: expand both parts of a file and append them in order into a single file with that name.

> [accordion] docsearch.py (part 1 of 2)
>
> ```python docsearch.py (part 1 of 2) theme={null}
> """Search a folder of Markdown files with pplx-embed-v2: the same model in dense and contextual mode, side by side.
>
> Usage:
>   python docsearch.py index <folder>              chunk every .md file, embed it three ways, write index/
>   python docsearch.py ask "<question>" [-k 3] [--configs NAME ...]   top chunks per configuration
>   python docsearch.py eval questions.jsonl         recall@k and MRR@100 for each configuration on labeled questions
> """
>
> import argparse
> import base64
> import json
> import os
> import re
> import sys
> import time
> from dataclasses import asdict, dataclass
> from pathlib import Path
>
> import httpx
> import numpy as np
>
> API_URL = "https://api.perplexity.ai/v1/embeddings"
> # One checkpoint per size serves both input modes. The index is always built with the 9b model; the 0.6b model shares
> # its vector space at 1024 dimensions, so it can embed questions against a 9b index built at 1024.
> INDEX_MODEL = "pplx-embed-v2-context-9b"
> SMALL_MODEL = "pplx-embed-v2-context-0.6b"
>
>
> @dataclass(frozen=True)
> class Config:
>     """One column of the evaluation: how the corpus was embedded and which model embeds the questions."""
>
>     name: str
>     mode: str  # "dense": every chunk embedded alone. "context": each document's chunks embedded together.
>     dimensions: int
>     query_model: str
>
>     @property
>     def index_key(self) -> str:
>         return f"{self.mode}-{self.dimensions}"
>
>
> CONFIGS = [
>     Config("dense-2048", "dense", 2048, INDEX_MODEL),
>     Config("context-2048", "context", 2048, INDEX_MODEL),
>     Config("context-1024", "context", 1024, INDEX_MODEL),
>     Config("context-1024-small", "context", 1024, SMALL_MODEL),
> ]
>
> # A chunk is one heading section. Longer sections are split at blank lines into pieces of about this size, and a single
> # line longer than this (an unwrapped paragraph, a minified file, an inline image) is cut into pieces of this size.
> CHUNK_CHARS = 1500
> # The API's caps are in tokens: 16,384 per text or per contextual document for the 9b model (32,768 for the 0.6b),
> # counting one extra token per chunk in contextual mode, and 120,000 per request. The two character budgets below are
> # heuristics tuned for this corpus, which runs about 3.6 characters per token overall and close to 3 in its link-heavy
> # release notes. A corpus near 2 characters per token (CJK text, minified code, long link lists) can exceed either cap
> # with these values; lower both, or count tokens with the published tokenizer.
> # A document is one file, unless the file is longer than this or has more than REQUEST_CHUNKS chunks, in which case its
> # sections are split into consecutive documents.
> DOC_CHARS = 40_000
> # Per request: 512 texts or documents, 16,000 chunks in a contextual request (the documented limit), and this many
> # characters of text as the budget for the 120,000-token cap.
> REQUEST_TEXTS = 512
> REQUEST_CHUNKS = 16_000
> REQUEST_CHARS = 240_000
> INDEX_DIR = Path("index")
> # How deep eval looks for the answer. A question whose answer ranks below this counts as not found.
> EVAL_DEPTH = 100
>
>
> # --- chunk
> @dataclass
> class Chunk:
>     doc: str  # document name: the file path, with #2, #3 for the later parts of a split file
>     heading: str
>     text: str
>
>
> def track_fence(line: str, fence: str) -> str:
>     """The code fence that is open after this line, or "" outside code. A fence opens with three or more backticks or
>     tildes and closes only on a line of at least as many of the same character, so a shorter fence nested inside a
>     longer one (a three-backtick Python block inside a four-backtick Markdown block) does not close the outer block."""
>     match = re.match(r" {0,3}(`{3,}|~{3,})(.*)", line)
>     if not match:
>         return fence
>     marker, rest = match.groups()
>     if not fence:
>         return marker
>     if marker[0] == fence[0] and len(marker) >= len(fence) and not rest.strip():
>         return ""
>     return fence
>
>
> def split_long(section: str) -> list[str]:
>     """Split one section at blank lines outside code fences into pieces of about CHUNK_CHARS. A paragraph or code block
>     longer than that on its own is split at line breaks, and a single line longer than that is cut into CHUNK_CHARS pieces."""
>     paragraphs, current, fence = [], "", ""
>     for line in section.splitlines(keepends=True):
>         fence = track_fence(line, fence)
>         if not line.strip() and not fence and current:
>             paragraphs.append(current.strip("\n"))
>             current = ""
>         else:
>             current += line
>     if current.strip():
>         paragraphs.append(current.strip("\n"))
>     pieces = []
>     for paragraph in paragraphs:
>         if len(paragraph) <= CHUNK_CHARS:
>             pieces.append(paragraph)
>         else:
>             lines = [line[i : i + CHUNK_CHARS] for line in paragraph.splitlines() for i in range(0, len(line) or 1, CHUNK_CHARS)]
>             pieces.extend(_group(lines, "\n"))
>     return _group(pieces, "\n\n")
>
>
> def _group(parts: list[str], sep: str) -> list[str]:
>     pieces, current = [], ""
>     for part in parts:
>         if current and len(current) + len(part) > CHUNK_CHARS:
>             pieces.append(current.strip())
>             current = ""
>         current += part + sep
>     if current.strip():
>         pieces.append(current.strip())
>     return pieces
>
>
> def chunk_markdown(text: str) -> list[tuple[str, str]]:
>     """Return (heading, chunk text) pairs. Each heading starts a chunk and stays in it; headings inside code fences do not count."""
>     sections: list[str] = []
>     fence = ""
>     for line in text.splitlines(keepends=True):
>         fence = track_fence(line, fence)
>         if sections and (fence or not re.match(r"#{1,6} ", line)):
>             sections[-1] += line
>         else:
>             sections.append(line)
>     chunks = []
>     heading = ""
>     for section in sections:
>         if not section.strip():
>             continue
>         if section.startswith("#"):
>             heading = section.splitlines()[0].lstrip("#").strip()
>         for piece in split_long(section):
>             chunks.append((heading, piece))
>     return chunks
>
>
> def load_folder(root: str) -> list[list[Chunk]]:
>     """One document per Markdown file, in path order. A file longer than DOC_CHARS or with more than REQUEST_CHUNKS
>     chunks becomes several documents, so that a document stays within the character budget for the per-document cap."""
>     documents = []
>     for path in sorted(Path(root).rglob("*.md")):
>         name = str(path.relative_to(root))
>         parts: list[list[Chunk]] = [[]]
>         for heading, text in chunk_markdown(path.read_text(encoding="utf-8")):
>             if parts[-1] and (len(parts[-1]) == REQUEST_CHUNKS or sum(len(c.text) for c in parts[-1]) + len(text) > DOC_CHARS):
>                 parts.append([])
>             parts[-1].append(Chunk(name, heading or path.name, text))
>         for i, part in enumerate(parts):
>             if part:
>                 if i:
>                     for chunk in part:
>                         chunk.doc = f"{name}#{i + 1}"
>                 documents.append(part)
>     return documents
>
>
> # --- embed
> def batches(items: list, size_of, limits: tuple[int, ...]):
>     """Yield consecutive slices whose summed sizes stay within limits. size_of returns one count per limit for an item.
>     An item that is over a limit on its own cannot be sent, so that is an error here rather than a rejected request."""
>     start, totals = 0, [0] * len(limits)
>     for i, item in enumerate(items):
>         sizes = size_of(item)
>         if any(s > lim for s, lim in zip(sizes, limits, strict=True)):
>             raise ValueError(f"item {i} alone measures {sizes}; the request limits are {limits}")
>         if i > start and any(t + s > lim for t, s, lim in zip(totals, sizes, limits)):
>             yield items[start:i]
>             start, totals = i, [0] * len(limits)
>         totals = [t + s for t, s in zip(totals, sizes)]
>     if start < len(items):
>         yield items[start:]
>
>
> def post(client: httpx.Client, body: dict) -> dict:
>     """One POST to /v1/embeddings. Retries 429 and 5xx with backoff; any other error stops with the API's message."""
>     for attempt in range(5):
>         response = client.post(API_URL, json=body)
>         if response.status_code == 429 or response.status_code >= 500:
>             time.sleep(2**attempt)
>             continue
>         if response.status_code != 200:
>             raise RuntimeError(f"HTTP {response.status_code} from the Embeddings API: {response.text}")
>         return response.json()
>     raise RuntimeError(f"HTTP {response.status_code} from the Embeddings API after 5 attempts: {response.text}")
>
>
> def decode(b64: str, dimensions: int) -> np.ndarray:
>     """A base64_int8 embedding as a signed int8 vector, checked against the dimensions the request asked for."""
>     vector = np.frombuffer(base64.b64decode(b64), dtype=np.int8)
>     if len(vector) != dimensions:
>         raise RuntimeError(f"the API returned a {len(vector)}-dimensional vector for a {dimensions}-dimensional request")
>     return vector
>
>
> def embed(client: httpx.Client, model: str, input_type: str, inputs: list, dimensions: int, usage: dict) -> np.ndarray:
>     """Int8 vectors, one row per text or per chunk, in input order. A flat list of strings is dense input: each text
>     is embedded alone. A list of lists is contextual input: each inner list is one document, and every chunk vector
>     reflects the rest of its document. input_type is "query" for questions and "document" for everything indexed."""
>     contextual = bool(inputs) and isinstance(inputs[0], list)
>     if contextual:
>         size_of = lambda doc: (1, len(doc), sum(len(c) for c in doc))
>         limits = (REQUEST_TEXTS, REQUEST_CHUNKS, REQUEST_CHARS)
>     else:
>         size_of = lambda text: (1, len(text))
>         limits = (REQUEST_TEXTS, REQUEST_CHARS)
>     rows = []
>     for batch in batches(inputs, size_of, limits):
>         body = {"model": model, "input_type": input_type, "input": batch, "dimensions": dimensions, "encoding_format": "base64_int8"}
>         response = post(client, body)
>         record(usage, model, response)
>         for item in in_order(response["data"]):
>             if contextual:
>                 rows.extend(decode(c["embedding"], dimensions) for c in in_order(item["data"]))
>             else:
>                 rows.append(decode(item["embedding"], dimensions))
>     return np.stack(rows)
>
>
> def in_order(items: list[dict]) -> list[dict]:
>     """Response items sorted by index, after checking that the indices are exactly 0..n-1, so a response with a missing
>     or repeated item cannot shift every row after it."""
>     ordered = sorted(items, key=lambda d: d["index"])
>     if [d["index"] for d in ordered] != list(range(len(ordered))):
>         raise RuntimeError(f"the API returned indices {[d['index'] for d in ordered]}; expected 0 to {len(ordered) - 1}")
>     return ordered
>
>
> ```

> [accordion] docsearch.py (part 2 of 2)
>
> ```python docsearch.py (part 2 of 2) theme={null}
> def record(usage: dict, model: str, response: dict) -> None:
>     """Add up requests, tokens, and cost per model. Cost is summed only when the response reports it."""
>     entry = usage.setdefault(model, {"requests": 0, "tokens": 0, "cost_usd": None})
>     entry["requests"] += 1
>     entry["tokens"] += response.get("usage", {}).get("total_tokens", 0)
>     cost = response.get("usage", {}).get("cost", {}).get("total_cost")
>     if cost is not None:
>         entry["cost_usd"] = (entry["cost_usd"] or 0.0) + cost
>
>
> # --- index
> @dataclass
> class Index:
>     """One int8 matrix (a row per chunk) and the float32 norm of each row, so cosine similarity is an integer dot
>     product divided by two norms. Storing int8 keeps 2,048 dimensions in 2 KB per chunk and 1,024 in 1 KB."""
>
>     matrix: np.ndarray
>     norms: np.ndarray
>
>     @classmethod
>     def of(cls, matrix: np.ndarray) -> "Index":
>         return cls(matrix, np.linalg.norm(matrix.astype(np.float32), axis=1))
>
>
> def build_index(client: httpx.Client, root: str, out: Path = INDEX_DIR) -> dict:
>     """Embed every chunk under root once per distinct (mode, dimensions) in CONFIGS, with INDEX_MODEL, and write the
>     int8 matrices, the chunk metadata, and a description of each matrix to out."""
>     documents = load_folder(root)
>     chunks = [chunk for doc in documents for chunk in doc]
>     usage: dict = {}
>     started = time.time()
>     matrices: dict[str, np.ndarray] = {}
>     for config in CONFIGS:
>         if config.index_key in matrices:
>             continue
>         if config.mode == "dense":
>             inputs: list = [c.text for c in chunks]
>         else:
>             inputs = [[c.text for c in doc] for doc in documents]
>         matrix = embed(client, INDEX_MODEL, "document", inputs, config.dimensions, usage)
>         if len(matrix) != len(chunks):
>             raise RuntimeError(f"{config.index_key}: embedding count {len(matrix)} does not match chunk count {len(chunks)}")
>         matrices[config.index_key] = matrix
>     out.mkdir(exist_ok=True)
>     np.savez(out / "vectors.npz", **matrices)
>     (out / "chunks.json").write_text(json.dumps([asdict(c) for c in chunks]))
>     described = {c.index_key: {"mode": c.mode, "dimensions": c.dimensions} for c in CONFIGS}
>     (out / "models.json").write_text(json.dumps({"model": INDEX_MODEL, "indexes": described}))
>     return {"documents": len(documents), "chunks": len(chunks), "seconds": time.time() - started, "usage": usage}
>
>
> def load_index(path: Path = INDEX_DIR) -> tuple[list[Chunk], dict[str, Index]]:
>     """Read the index back. Refuses an index built with another model, one that lacks a matrix a config needs, or one
>     whose saved vectors do not have one row per chunk and the dimensions recorded when it was built."""
>     models = json.loads((path / "models.json").read_text())
>     if models["model"] != INDEX_MODEL:
>         raise RuntimeError(f"index was built with {models['model']}; run index again")
>     missing = sorted({c.index_key for c in CONFIGS} - set(models["indexes"]))
>     if missing:
>         raise RuntimeError(f"index has no vectors for {', '.join(missing)}; run index again")
>     chunks = [Chunk(**c) for c in json.loads((path / "chunks.json").read_text())]
>     vectors = np.load(path / "vectors.npz")
>     indexes = {}
>     for key, described in models["indexes"].items():
>         matrix = vectors[key]
>         if matrix.dtype != np.int8 or matrix.ndim != 2 or matrix.shape[1] != described["dimensions"]:
>             raise RuntimeError(f"{key} vectors are {matrix.dtype} with shape {matrix.shape}, models.json says int8 with {described['dimensions']} dimensions; run index again")
>         if matrix.shape[0] != len(chunks):
>             raise RuntimeError(f"{key} has {matrix.shape[0]} vectors for {len(chunks)} chunks; run index again")
>         indexes[key] = Index.of(matrix)
>     return chunks, indexes
>
>
> def embed_queries(client: httpx.Client, questions: list[str]) -> dict[str, np.ndarray]:
>     """One query matrix per config, embedded with the config's query model at the config's dimensions. Questions go
>     as flat strings with input_type "query": a question has no surrounding document."""
>     usage: dict = {}
>     queries = {}
>     for config in CONFIGS:
>         queries[config.name] = embed(client, config.query_model, "query", questions, config.dimensions, usage)
>     return queries
>
>
> def rank(index: Index, query: np.ndarray) -> np.ndarray:
>     """Row indices from most to least similar, ranked by cosine: the dot product over the query's norm times the row's
>     norm, with the query's norm left out because it is the same for every row. The dot product runs in int32 (int8
>     times int8 summed over 2,048 dimensions cannot overflow it) and the row norms divide it in float32."""
>     if query.shape[-1] != index.matrix.shape[1]:
>         raise RuntimeError(f"query vector has {query.shape[-1]} dimensions but the index has {index.matrix.shape[1]}; run index again")
>     scores = (index.matrix @ query.astype(np.int32)) / index.norms
>     return np.argsort(-scores, kind="stable")
>
>
> # --- ask
> def ask(client: httpx.Client, question: str, k: int, names: list[str]) -> None:
>     chunks, indexes = load_index()
>     queries = embed_queries(client, [question])
>     for config in [c for c in CONFIGS if c.name in names]:
>         print(f"\n{config.name}: {config.mode} index at {config.dimensions} dimensions, question embedded by {config.query_model}")
>         for place, row in enumerate(rank(indexes[config.index_key], queries[config.name][0])[:k], start=1):
>             chunk = chunks[row]
>             preview = " ".join(chunk.text.split())[:160].rstrip()
>             print(f"  {place}. {chunk.doc} > {chunk.heading}\n     {preview}")
>
>
> # --- eval
> def phrases(label: dict) -> list[str]:
>     """The `contains` field is one phrase or a list of phrases; a chunk that has any of them is accepted."""
>     value = label["contains"]
>     return [value] if isinstance(value, str) else list(value)
>
>
> def hit(chunk: Chunk, label: dict) -> bool:
>     """A chunk answers a question when it comes from the labeled file and contains one of the labeled phrases. Only the
>     #2, #3 part suffix of a split file is removed before comparing, so a file named `C# guide.md` keeps its name."""
>     text = chunk.text.lower()
>     return re.sub(r"#\d+$", "", chunk.doc) == label["file"] and any(p.lower() in text for p in phrases(label) if p.strip())
>
>
> def check_labels(chunks: list[Chunk], labels: list[dict]) -> None:
>     """Stop before spending any tokens on a label file that cannot be scored: an empty phrase would accept every chunk
>     of a file, a repeated question would count twice, and a label that matches no chunk is a typo in the file or phrase,
>     or a corpus that changed since the label was written."""
>     if any(not phrases(label) or not all(p.strip() for p in phrases(label)) for label in labels):
>         sys.exit("every label needs at least one non-empty phrase in 'contains'")
>     questions = [label["question"] for label in labels]
>     if len(set(questions)) != len(questions):
>         sys.exit("duplicate question in the label file: " + next(q for q in questions if questions.count(q) > 1))
>     unmatched = [label for label in labels if not any(hit(chunk, label) for chunk in chunks)]
>     if unmatched:
>         lines = [f"  {label['file']} containing {label['contains']!r}: {label['question']}" for label in unmatched]
>         sys.exit(f"{len(unmatched)} label(s) match no chunk in the index. Fix the label or rebuild the index.\n" + "\n".join(lines))
>
>
> def evaluate(chunks: list[Chunk], indexes: dict[str, Index], queries: dict, labels: list[dict]) -> dict[str, list[int]]:
>     """For each config, the rank of the first chunk that answers each question (0 when none in the top EVAL_DEPTH)."""
>     ranks: dict[str, list[int]] = {}
>     for config in CONFIGS:
>         ranks[config.name] = []
>         for label, query in zip(labels, queries[config.name], strict=True):
>             order = rank(indexes[config.index_key], query)[:EVAL_DEPTH]
>             found = next((i + 1 for i, row in enumerate(order) if hit(chunks[row], label)), 0)
>             ranks[config.name].append(found)
>     return ranks
>
>
> def summarize(ranks: list[int]) -> dict[str, float]:
>     """recall@1, 3, 5 and MRR over the top EVAL_DEPTH, so the metric is MRR@100 and an answer below that counts as zero."""
>     found = [r for r in ranks if r]
>     result = {f"recall@{k}": sum(1 for r in found if r <= k) / len(ranks) for k in (1, 3, 5)}
>     result[f"MRR@{EVAL_DEPTH}"] = sum(1 / r for r in found) / len(ranks)
>     return result
>
>
> # --- output
> def print_eval(labels: list[dict], ranks: dict[str, list[int]]) -> None:
>     names = [c.name for c in CONFIGS]
>     width = max(len(n) for n in names)
>     metrics = {name: summarize(ranks[name]) for name in names}
>     print(f"{len(labels)} questions\n")
>     print(f"{'configuration':<{width}}  " + "  ".join(f"{k:>8}" for k in metrics[names[0]]))
>     for name in names:
>         print(f"{name:<{width}}  " + "  ".join(f"{v:>8.3f}" for v in metrics[name].values()))
>     print(f"\nRank of the first chunk that answers each question (1 is best, - means not in the top {EVAL_DEPTH})")
>     print(f"{'question':<62}  " + "  ".join(f"{n:>{len(n)}}" for n in names))
>     for i, label in enumerate(labels):
>         print(f"{label['question'][:62]:<62}  " + "  ".join(f"{ranks[n][i] or '-':>{len(n)}}" for n in names))
>
>
> def main(argv: list[str] | None = None) -> None:
>     parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
>     sub = parser.add_subparsers(dest="command", required=True)
>     sub.add_parser("index").add_argument("folder")
>     p_ask = sub.add_parser("ask")
>     p_ask.add_argument("question")
>     p_ask.add_argument("-k", type=int, default=3)
>     p_ask.add_argument("--configs", nargs="+", choices=[c.name for c in CONFIGS], default=["dense-2048", "context-2048"])
>     sub.add_parser("eval").add_argument("questions")
>     args = parser.parse_args(argv)
>     key = os.environ.get("PERPLEXITY_API_KEY")
>     if not key:
>         sys.exit("export PERPLEXITY_API_KEY before running this script")
>     client = httpx.Client(headers={"Authorization": f"Bearer {key}"}, timeout=120)
>
>     if args.command == "index":
>         report = build_index(client, args.folder)
>         print(f"{report['documents']} documents, {report['chunks']} chunks, indexed in {report['seconds']:.0f} s")
>         for model, entry in report["usage"].items():
>             cost = f"${entry['cost_usd']:.4f}" if entry["cost_usd"] is not None else "cost not reported by the API"
>             print(f"  {model}: {entry['requests']} requests, {entry['tokens']:,} tokens, {cost}")
>     elif args.command == "ask":
>         ask(client, args.question, args.k, args.configs)
>     else:
>         labels = [json.loads(line) for line in Path(args.questions).read_text().splitlines() if line.strip()]
>         chunks, indexes = load_index()
>         check_labels(chunks, labels)
>         queries = embed_queries(client, [label["question"] for label in labels])
>         print_eval(labels, evaluate(chunks, indexes, queries, labels))
>
>
> if __name__ == "__main__":
>     main()
> ```

> [accordion] test\_docsearch.py (part 1 of 2)
>
> ```python test_docsearch.py (part 1 of 2) theme={null}
> """Offline tests for docsearch.py. A fake Embeddings API behind an httpx transport stands in for the service, so no
> key is needed. The fake reproduces the behavior recorded from the live API: input_type is required, flat input is dense
> and nested input is contextual, each model accepts its own dimensions, vectors come back as base64 int8, and the error
> wording is the recorded one. Any other check it makes is the recipe's own."""
>
> import base64
> import json
> import random
>
> import httpx
> import numpy as np
> import pytest
>
> import docsearch as ds
>
>
> def test_chunk_markdown_starts_a_chunk_at_every_heading_and_keeps_the_heading_line():
>     text = "# Title\n\nIntro.\n\n## Run it\n\nRun `pytest`.\n\n### Details\n\nMore.\n"
>     chunks = ds.chunk_markdown(text)
>     assert [h for h, _ in chunks] == ["Title", "Run it", "Details"]
>     assert chunks[1][1] == "## Run it\n\nRun `pytest`."
>
>
> def test_chunk_markdown_ignores_headings_and_blank_lines_inside_code_fences(monkeypatch):
>     fence = "`" * 3
>     chunks = ds.chunk_markdown(f"## Setup\n\n{fence}python\n# not a heading\nx = 1\n{fence}\n\n## Next\n\nText.\n")
>     assert [h for h, _ in chunks] == ["Setup", "Next"]
>     assert "# not a heading" in chunks[0][1]
>     monkeypatch.setattr(ds, "CHUNK_CHARS", 40)
>     chunks = ds.chunk_markdown("## Code\n\n" + "p" * 30 + f"\n\n{fence}\na\n\nb\n{fence}\n")
>     assert [text for _, text in chunks] == ["## Code\n\n" + "p" * 30, f"{fence}\na\n\nb\n{fence}"]
>
>
> def test_chunk_markdown_tracks_the_fence_character_and_length():
>     outer, inner = "`" * 4, "`" * 3
>     text = f"# Writing Markdown\n\n{outer}markdown\n{inner}python\n# a comment, not a heading\nprint(1)\n{inner}\n{outer}\n\n## Next\n\nBody.\n"
>     assert [h for h, _ in ds.chunk_markdown(text)] == ["Writing Markdown", "Next"]
>     tildes = "~" * 3
>     text = f"# Mixed\n\n{tildes}\n{inner}\n# still code\n{tildes}\n\n## After\n\nBody.\n"
>     assert [h for h, _ in ds.chunk_markdown(text)] == ["Mixed", "After"]
>
>
> def test_chunk_markdown_splits_long_sections_at_blank_lines(monkeypatch):
>     monkeypatch.setattr(ds, "CHUNK_CHARS", 60)
>     paragraphs = [f"Paragraph {i} " + "x" * 30 for i in range(4)]
>     chunks = ds.chunk_markdown("## Long\n\n" + "\n\n".join(paragraphs))
>     assert len(chunks) == 4
>     assert all(heading == "Long" for heading, _ in chunks)
>     assert chunks[0][1].startswith("## Long\n\nParagraph 0")
>     assert all("\n\nParagraph" not in text or len(text) <= 60 for _, text in chunks)
>
>
> def test_chunk_markdown_splits_an_oversized_paragraph_at_line_breaks(monkeypatch):
>     monkeypatch.setattr(ds, "CHUNK_CHARS", 50)
>     lines = "\n".join(f"* item {i} " + "y" * 20 for i in range(6))
>     chunks = ds.chunk_markdown("## List\n\n" + lines)
>     assert len(chunks) > 1
>     assert "".join(text for _, text in chunks).count("* item") == 6
>
>
> def test_chunk_markdown_cuts_a_single_line_longer_than_the_chunk_size(monkeypatch):
>     monkeypatch.setattr(ds, "CHUNK_CHARS", 1000)
>     chunks = ds.chunk_markdown("# Overview\n\n" + "operation " * 40_000)
>     assert all(len(text) <= 1000 for _, text in chunks)
>     assert "".join(text for _, text in chunks).count("operation") == 40_000
>     image = "![screenshot](data:image/png;base64," + "aB9+" * 100_000 + ")"
>     chunks = ds.chunk_markdown("# Screenshot\n\n" + image + "\n\n## Next\n\nBody.\n")
>     assert all(len(text) <= 1000 for _, text in chunks)
>     assert chunks[-1] == ("Next", "## Next\n\nBody.")
>
>
> def test_load_folder_makes_one_document_per_file_and_splits_long_files(tmp_path, monkeypatch):
>     monkeypatch.setattr(ds, "DOC_CHARS", 120)
>     (tmp_path / "a.md").write_text("# A\n\nShort file.\n")
>     (tmp_path / "sub").mkdir()
>     (tmp_path / "sub" / "b.md").write_text("".join(f"## Part {i}\n\n{'z' * 40}\n\n" for i in range(4)))
>     documents = ds.load_folder(str(tmp_path))
>     names = [doc[0].doc for doc in documents]
>     assert names == ["a.md", "sub/b.md", "sub/b.md#2"]
>     assert all(sum(len(c.text) for c in doc) <= 120 for doc in documents[1:])
>     assert sum(len(doc) for doc in documents) == 5
>
>
> def test_load_folder_splits_a_file_with_many_short_sections_so_a_document_fits_one_request(tmp_path, monkeypatch):
>     monkeypatch.setattr(ds, "REQUEST_CHUNKS", 200)
>     (tmp_path / "options.md").write_text("".join(f"## Option {i}\n\nValue.\n\n" for i in range(450)))
>     documents = ds.load_folder(str(tmp_path))
>     assert [len(doc) for doc in documents] == [200, 200, 50]
>     assert [doc[0].doc for doc in documents] == ["options.md", "options.md#2", "options.md#3"]
>     size_of = lambda doc: (1, len(doc), sum(len(c.text) for c in doc))
>     assert all(len(b) == 1 for b in ds.batches(documents, size_of, (ds.REQUEST_TEXTS, 200, ds.REQUEST_CHARS)))
>
>
> def test_batches_respect_every_limit_and_refuse_an_item_that_cannot_fit():
>     items = ["a" * 10, "b" * 10, "c" * 25, "d" * 10, "e" * 10, "f" * 10]
>     by_count = list(ds.batches(items, lambda t: (1, len(t)), (2, 1000)))
>     assert [len(b) for b in by_count] == [2, 2, 2]
>     by_chars = list(ds.batches(items, lambda t: (1, len(t)), (100, 25)))
>     assert by_chars == [["a" * 10, "b" * 10], ["c" * 25], ["d" * 10, "e" * 10], ["f" * 10]]
>     with pytest.raises(ValueError, match=r"item 2 alone measures \(1, 25\); the request limits are \(100, 20\)"):
>         list(ds.batches(items, lambda t: (1, len(t)), (100, 20)))
>     documents = [["x"], ["y", "z"], ["w"]]
>     size_of = lambda d: (1, len(d), sum(len(c) for c in d))
>     assert list(ds.batches(documents, size_of, (10, 2, 100))) == [[["x"]], [["y", "z"]], [["w"]]]
>     with pytest.raises(ValueError, match=r"measures \(1, 2, 2\)"):
>         list(ds.batches(documents, size_of, (10, 1, 100)))
>
>
> class FakeEmbeddingsAPI:
>     """POST /v1/embeddings with the request shape and response shape the recipe relies on. The error messages for
>     input_type, dimensions, and encoding_format are the live API's wording; the other checks are this fake's own, not
>     verified API behavior. Vectors are deterministic: the first value is the text length,
>     the second is 1 for dense and 2 for contextual input, the rest are -1, at the requested dimensions. Both models
>     return the same vector for the same text, which is the shared-space property the cross-size config relies on.
>     Items are shuffled in the response so the code under test has to restore input order."""
>
>     DIMENSIONS = {"pplx-embed-v2-context-9b": (1024, 2048), "pplx-embed-v2-context-0.6b": (512, 1024)}
>
>     def __init__(self, fail_first_with: int | None = None):
>         self.bodies: list[dict] = []
>         self.fail_first_with = fail_first_with
>         self.transport = httpx.MockTransport(self.handle)
>
>     def client(self) -> httpx.Client:
>         return httpx.Client(transport=self.transport, headers={"Authorization": "Bearer test-key"})
>
>     def handle(self, request: httpx.Request) -> httpx.Response:
>         if request.url.path != "/v1/embeddings" or not request.headers.get("Authorization", "").startswith("Bearer "):
>             return httpx.Response(401, json={"error": "unauthorized"})
>         if self.fail_first_with:
>             status, self.fail_first_with = self.fail_first_with, None
>             return httpx.Response(status, json={"error": "try again"})
>         body = json.loads(request.content)
>         self.bodies.append(body)
>         error = self.validate(body)
>         if error:
>             return httpx.Response(400, json={"error": error})
>         dims = body.get("dimensions") or self.DIMENSIONS[body["model"]][-1]
>         inputs = body["input"]
>         if isinstance(inputs[0], str):
>             data = [{"index": i, "embedding": self.vector(t, 1, dims)} for i, t in enumerate(inputs)]
>             tokens = sum(len(t) for t in inputs)
>         else:
>             data = []
>             for i, doc in enumerate(inputs):
>                 inner = [{"index": j, "embedding": self.vector(c, 2, dims)} for j, c in enumerate(doc)]
>                 random.shuffle(inner)
>                 data.append({"index": i, "data": inner})
>             tokens = sum(len(c) for doc in inputs for c in doc)
>         random.shuffle(data)
>         usage = {"prompt_tokens": tokens, "total_tokens": tokens, "cost": {"total_cost": tokens / 1e6}}
>         return httpx.Response(200, json={"data": data, "model": body["model"], "usage": usage})
>
>     def validate(self, body: dict) -> str | None:
>         if body.get("model") not in self.DIMENSIONS:
>             return "invalid model"
>         if body.get("input_type") not in ("query", "document"):
>             return f"input_type is required for model {body.get('model')}; use query or document"
>         if "num_chunks" in body:
>             return "num_chunks is only supported for image input"
>         if body.get("encoding_format", "base64_int8") not in ("base64_int8", "base64_binary"):
>             return "encoding format must be one of: base64_int8 base64_binary"
>         if body.get("dimensions") is not None and body["dimensions"] not in self.DIMENSIONS[body["model"]]:
>             return f"Output dimensions must be one of {list(self.DIMENSIONS[body['model']])}, got {body['dimensions']}."
>         inputs = body.get("input")
>         if not isinstance(inputs, list) or not inputs or len(inputs) > 512:
>             return "input must be a non-empty list of at most 512 items"
>         if all(isinstance(x, str) for x in inputs):
>             return "empty text" if any(not x for x in inputs) else None
>         if not all(isinstance(x, list) and x and all(isinstance(c, str) and c for c in x) for x in inputs):
>             return "input must be a list of strings or a list of lists of strings"
>         if sum(len(x) for x in inputs) > 16_000:
>             return "too many chunks in one contextual request"
>         return None
>
>     @staticmethod
>     def vector(text: str, mode: int, dims: int) -> str:
>         values = np.full(dims, -1, dtype=np.int8)
>         values[0], values[1] = min(len(text), 127), mode
>         return base64.b64encode(values.tobytes()).decode()
>
>
> def test_fake_api_rejects_malformed_requests_and_post_surfaces_the_message():
>     api = FakeEmbeddingsAPI()
>     good = {"model": ds.INDEX_MODEL, "input_type": "document", "input": ["a"], "dimensions": 1024, "encoding_format": "base64_int8"}
>     assert len(ds.post(api.client(), good)["data"]) == 1
>     for change, message in [
>         ({"input_type": None}, "input_type is required"),
>         ({"input": ["a", ["b"]]}, "list of strings or a list of lists"),
>         ({"dimensions": 512}, r"Output dimensions must be one of \[1024, 2048\], got 512"),
>         ({"model": ds.SMALL_MODEL, "dimensions": 2048}, r"Output dimensions must be one of \[512, 1024\], got 2048"),
>         ({"encoding_format": "float"}, "encoding format must be one of: base64_int8 base64_binary"),
>         ({"input": ["a", ""]}, "empty text"),
>         ({"num_chunks": 4}, "only supported for image input"),
>     ]:
>         with pytest.raises(RuntimeError, match=f"HTTP 400 from the Embeddings API: .*{message}"):
>             ds.post(api.client(), {**good, **change})
>
>
> ```

> [accordion] test\_docsearch.py (part 2 of 2)
>
> ```python test_docsearch.py (part 2 of 2) theme={null}
> def test_post_retries_429_and_5xx_then_gives_up(monkeypatch):
>     monkeypatch.setattr(ds.time, "sleep", lambda s: None)
>     body = {"model": ds.INDEX_MODEL, "input_type": "query", "input": ["a"], "dimensions": 1024}
>     assert len(ds.post(FakeEmbeddingsAPI(fail_first_with=429).client(), body)["data"]) == 1
>     assert len(ds.post(FakeEmbeddingsAPI(fail_first_with=503).client(), body)["data"]) == 1
>     always = httpx.Client(transport=httpx.MockTransport(lambda r: httpx.Response(429, text="slow down")))
>     with pytest.raises(RuntimeError, match="HTTP 429 from the Embeddings API after 5 attempts: slow down"):
>         ds.post(always, body)
>
>
> def test_decode_returns_signed_int8_and_checks_the_length():
>     vector = ds.decode(base64.b64encode(np.array([127, -128, 0, -1], dtype=np.int8).tobytes()).decode(), 4)
>     assert vector.dtype == np.int8 and vector.tolist() == [127, -128, 0, -1]
>     with pytest.raises(RuntimeError, match="returned a 4-dimensional vector for a 1024-dimensional request"):
>         ds.decode(base64.b64encode(bytes(4)).decode(), 1024)
>
>
> def test_embed_sends_input_type_and_dimensions_and_parses_both_response_shapes_in_input_order(monkeypatch):
>     monkeypatch.setattr(ds, "REQUEST_TEXTS", 3)
>     monkeypatch.setattr(ds, "REQUEST_CHUNKS", 3)
>     api, usage = FakeEmbeddingsAPI(), {}
>     texts = ["a", "bb", "ccc", "dddd", "eeeee"]
>     dense = ds.embed(api.client(), ds.INDEX_MODEL, "document", texts, 2048, usage)
>     assert dense.dtype == np.int8 and dense.shape == (5, 2048)
>     assert dense[:, 0].tolist() == [1, 2, 3, 4, 5] and set(dense[:, 1].tolist()) == {1}
>     contextual = ds.embed(api.client(), ds.INDEX_MODEL, "document", [["a", "bb"], ["ccc"], ["dddd", "eeeee"]], 1024, usage)
>     assert contextual.shape == (5, 1024) and contextual[:, 0].tolist() == [1, 2, 3, 4, 5] and set(contextual[:, 1].tolist()) == {2}
>     one_chunk = ds.embed(api.client(), ds.INDEX_MODEL, "document", [["only"]], 1024, usage)
>     assert one_chunk.shape == (1, 1024) and one_chunk[0, 1] == 2
>     queries = ds.embed(api.client(), ds.SMALL_MODEL, "query", ["what?"], 1024, usage)
>     assert queries.shape == (1, 1024)
>     assert [b["input_type"] for b in api.bodies] == ["document"] * 5 + ["query"]
>     assert all(b["encoding_format"] == "base64_int8" and "dimensions" in b for b in api.bodies)
>     assert [len(b["input"]) for b in api.bodies] == [3, 2, 2, 1, 1, 1]
>     assert all(sum(len(d) for d in b["input"]) <= 3 for b in api.bodies if isinstance(b["input"][0], list))
>     assert usage[ds.INDEX_MODEL] == {"requests": 5, "tokens": 15 + 15 + 4, "cost_usd": pytest.approx(34 / 1e6)}
>     assert usage[ds.SMALL_MODEL]["requests"] == 1
>
>
> def test_embed_refuses_a_response_whose_indices_are_not_exactly_0_to_n_minus_1(monkeypatch):
>     api = FakeEmbeddingsAPI()
>     original = api.handle
>
>     def shift_one(request):
>         response = original(request)
>         data = response.json()["data"]
>         next(d for d in data if d["index"] == 1)["index"] = 2  # a shifted index must not silently move every later row
>         return httpx.Response(200, json={**response.json(), "data": data})
>
>     monkeypatch.setattr(api, "transport", httpx.MockTransport(shift_one))
>     with pytest.raises(RuntimeError, match=r"the API returned indices \[0, 2\]; expected 0 to 1"):
>         ds.embed(api.client(), ds.INDEX_MODEL, "document", ["a", "b"], 2048, {})
>
>
> def test_build_index_stores_int8_matrices_per_configuration_and_load_index_checks_them(tmp_path, monkeypatch):
>     (tmp_path / "docs").mkdir()
>     (tmp_path / "docs" / "a.md").write_text("# A\n\nOne.\n\n## B\n\nTwo.\n")
>     api = FakeEmbeddingsAPI()
>     report = ds.build_index(api.client(), str(tmp_path / "docs"), out=tmp_path / "index")
>     assert [(b["input_type"], b["dimensions"], isinstance(b["input"][0], list)) for b in api.bodies] == [("document", 2048, False), ("document", 2048, True), ("document", 1024, True)]
>     assert all(b["model"] == ds.INDEX_MODEL for b in api.bodies)
>     chunks, indexes = ds.load_index(tmp_path / "index")
>     assert report["chunks"] == len(chunks) == 2
>     assert set(indexes) == {"dense-2048", "context-2048", "context-1024"}
>     assert indexes["dense-2048"].matrix.dtype == np.int8 and indexes["dense-2048"].matrix.shape == (2, 2048)
>     assert indexes["context-1024"].matrix.shape == (2, 1024) and indexes["context-1024"].norms.dtype == np.float32
>     saved = dict(np.load(tmp_path / "index" / "vectors.npz"))
>     assert all(m.dtype == np.int8 for m in saved.values()) and sum(m.nbytes for m in saved.values()) == 2 * (2048 + 2048 + 1024)
>     models = json.loads((tmp_path / "index" / "models.json").read_text())
>     assert models == {"model": ds.INDEX_MODEL, "indexes": {"dense-2048": {"mode": "dense", "dimensions": 2048}, "context-2048": {"mode": "context", "dimensions": 2048}, "context-1024": {"mode": "context", "dimensions": 1024}}}
>     broken = {**models, "indexes": {**models["indexes"], "context-1024": {"mode": "context", "dimensions": 2048}}}
>     (tmp_path / "index" / "models.json").write_text(json.dumps(broken))
>     with pytest.raises(RuntimeError, match=r"context-1024 vectors are int8 with shape \(2, 1024\), models.json says int8 with 2048 dimensions"):
>         ds.load_index(tmp_path / "index")
>     (tmp_path / "index" / "models.json").write_text(json.dumps({**models, "indexes": {k: v for k, v in models["indexes"].items() if k != "dense-2048"}}))
>     with pytest.raises(RuntimeError, match="index has no vectors for dense-2048; run index again"):
>         ds.load_index(tmp_path / "index")
>     (tmp_path / "index" / "models.json").write_text(json.dumps({**models, "model": "pplx-embed-v1-4b"}))
>     with pytest.raises(RuntimeError, match="index was built with pplx-embed-v1-4b; run index again"):
>         ds.load_index(tmp_path / "index")
>
>
> def test_load_index_refuses_a_matrix_with_the_wrong_row_count(tmp_path):
>     (tmp_path / "docs").mkdir()
>     (tmp_path / "docs" / "a.md").write_text("# A\n\nOne.\n\n## B\n\nTwo.\n")
>     ds.build_index(FakeEmbeddingsAPI().client(), str(tmp_path / "docs"), out=tmp_path / "index")
>     saved = dict(np.load(tmp_path / "index" / "vectors.npz"))
>     np.savez(tmp_path / "index" / "vectors.npz", **{**saved, "context-2048": saved["context-2048"][:1]})
>     with pytest.raises(RuntimeError, match="context-2048 has 1 vectors for 2 chunks; run index again"):
>         ds.load_index(tmp_path / "index")
>
>
> def test_small_model_queries_the_1024_index_and_dimension_mismatches_are_refused(tmp_path):
>     (tmp_path / "docs").mkdir()
>     (tmp_path / "docs" / "a.md").write_text("# A\n\nOne.\n\n## B\n\nTwo.\n")
>     api = FakeEmbeddingsAPI()
>     ds.build_index(api.client(), str(tmp_path / "docs"), out=tmp_path / "index")
>     chunks, indexes = ds.load_index(tmp_path / "index")
>     queries = ds.embed_queries(api.client(), ["which?"])
>     query_bodies = api.bodies[3:]
>     assert [(b["model"], b["dimensions"], b["input_type"]) for b in query_bodies] == [(ds.INDEX_MODEL, 2048, "query"), (ds.INDEX_MODEL, 2048, "query"), (ds.INDEX_MODEL, 1024, "query"), (ds.SMALL_MODEL, 1024, "query")]
>     small = queries["context-1024-small"][0]
>     assert small.shape == (1024,) and small.dtype == np.int8
>     assert ds.rank(indexes["context-1024"], small).tolist() == ds.rank(indexes["context-1024"], queries["context-1024"][0]).tolist()
>     with pytest.raises(RuntimeError, match="query vector has 1024 dimensions but the index has 2048; run index again"):
>         ds.rank(indexes["context-2048"], small)
>
>
> def test_rank_is_cosine_over_int8_rows():
>     matrix = np.array([[100, 0], [60, 60], [-100, 0]], dtype=np.int8)
>     index = ds.Index.of(matrix)
>     assert index.norms.dtype == np.float32
>     assert ds.rank(index, np.array([1, 1], dtype=np.int8)).tolist() == [1, 0, 2]
>     assert ds.rank(index, np.array([127, 0], dtype=np.int8)).tolist() == [0, 1, 2]
>     big = ds.Index.of(np.full((1, 2048), 127, dtype=np.int8))
>     assert ds.rank(big, np.full(2048, 127, dtype=np.int8)).tolist() == [0]
>
>
> def test_evaluate_reports_rank_of_first_hit_counts_split_files_and_zero_when_missing():
>     chunks = [ds.Chunk("a.md", "A", "about pytest"), ds.Chunk("b.md", "B", "pytest again"), ds.Chunk("a.md#2", "C", "pytest too")]
>     eye = np.eye(3, dtype=np.int8) * 100
>     indexes = {"dense-2048": ds.Index.of(eye), "context-2048": ds.Index.of(eye[[2, 1, 0]]), "context-1024": ds.Index.of(eye)}
>     labels = [
>         {"question": "q1", "file": "a.md", "contains": "PYTEST"},
>         {"question": "q2", "file": "a.md", "contains": "too"},
>         {"question": "q3", "file": "c.md", "contains": "x"},
>     ]
>     queries = {c.name: np.array([[100, 50, 0], [0, 0, 100], [0, 100, 0]], dtype=np.int8) for c in ds.CONFIGS}
>     ranks = ds.evaluate(chunks, indexes, queries, labels)
>     assert ranks == {"dense-2048": [1, 1, 0], "context-2048": [1, 3, 0], "context-1024": [1, 1, 0], "context-1024-small": [1, 1, 0]}
>     assert ds.summarize([1, 0, 3, 4]) == {"recall@1": 0.25, "recall@3": 0.5, "recall@5": 0.75, "MRR@100": pytest.approx((1 + 1 / 3 + 1 / 4) / 4)}
>
>
> def test_hit_keeps_a_hash_in_a_real_file_name_and_strips_only_the_part_suffix():
>     label = {"question": "How do I print?", "file": "C# guide.md", "contains": "Console.WriteLine"}
>     assert ds.hit(ds.Chunk("C# guide.md", "Output", "Use Console.WriteLine."), label)
>     assert ds.hit(ds.Chunk("C# guide.md#2", "Output", "Use Console.WriteLine."), label)
>     assert not ds.hit(ds.Chunk("C.md", "Output", "Use Console.WriteLine."), label)
>
>
> def test_check_labels_exits_before_any_api_call_when_a_label_matches_nothing():
>     chunks = [ds.Chunk("a.md", "Tests", "Run pytest.")]
>     ds.check_labels(chunks, [{"question": "q1", "file": "a.md", "contains": "pytest"}])
>     with pytest.raises(SystemExit) as stop:
>         ds.check_labels(chunks, [{"question": "q2", "file": "a.md", "contains": "unittest"}, {"question": "q3", "file": "b.md", "contains": "pytest"}])
>     assert "2 label(s) match no chunk" in str(stop.value) and "b.md containing 'pytest': q3" in str(stop.value)
>
>
> def test_contains_accepts_a_list_of_phrases_and_rejects_empty_or_duplicate_labels():
>     chunks = [ds.Chunk("a.md", "Start", "Use the lifespan parameter."), ds.Chunk("a.md", "Old", 'Or @app.on_event("startup").')]
>     label = {"question": "q1", "file": "a.md", "contains": ["lifespan", '@app.on_event("startup")']}
>     assert all(ds.hit(chunk, label) for chunk in chunks)
>     assert not ds.hit(ds.Chunk("a.md", "Other", "Nothing here."), label)
>     assert not ds.hit(chunks[0], {"question": "q1", "file": "a.md", "contains": ""})
>     ds.check_labels(chunks, [label])
>     for bad in ([], [""], " ", ["lifespan", ""]):
>         with pytest.raises(SystemExit, match="non-empty phrase"):
>             ds.check_labels(chunks, [{"question": "q1", "file": "a.md", "contains": bad}])
>     with pytest.raises(SystemExit, match="duplicate question in the label file: q1"):
>         ds.check_labels(chunks, [label, {"question": "q1", "file": "a.md", "contains": "lifespan"}])
> ```