What the numbers mean

Documents and execution

Four synthetic, single-page PDFs cover ordinary text, a regular financial table, a table with blank cells, and two-column reading order. The source files and SHA-256 hashes are published with the results. Each free adapter runs sequentially three times; we report the median and every sample. These are warm-process local measurements, not cloud latency promises. Failures remain visible and stop further samples for that combination.

Cell alignment

The evaluator looks for a Markdown pipe table with exactly the reference headers. It matches unique row labels, then compares every non-label cell against its expected column, including blank cells. Case, whitespace and bold markers are normalised. Numeric punctuation is preserved. A matching value under the wrong column receives no credit. The percentage is matched cells divided by expected cells.

A text extractor may preserve all words and still score zero on this structural measure. That describes the output format, not a broken text extractor. HTML tables and alternative equivalent layouts are not recognised by this evaluator. Read the raw output alongside the number.

Reading order

The two-column fixture contains twenty numbered lines in each column. The check requires every left-column line, in sequence, followed by every right-column line. It also reports how many expected lines are present. This is one controlled layout, not a general document-order score.

RAG comparison

Live Q&A uses one chunker, one embedding model, one question vector, top-5 retrieval and the same answer prompt across parsers. Evidence is shown with each answer. The answer model can still vary at temperature zero, so disagreement is an invitation to inspect the source, not proof of which answer is right. We publish no automated answer-accuracy score.

Costs and exclusions

Measured tokens and retrieval units are priced using the deployment’s configured rates. Write units are estimated from record sizes. Missing rates appear as unpriced. Storage, egress, provider minimums, and failed or interrupted requests can add charges. Hosted Hyperbots OCR is excluded from this free benchmark, and these fixtures do not evaluate scans, handwriting, encrypted PDFs or multilingual extraction.

Repeat the experiment

In a checkout with web and engine dependencies installed, run npm run benchmark. This refreshes the results, source hashes and downloadable fixtures without using account credentials. The fixture generator and reference cells are versioned with the project.