The Two Best AI Code Reviewers Score the Same. One Costs $1.43 a Run, the Other $9.05.

Written with AI assistance. Figures without a traceable source were cut before publishing.

On ReactBench, GPT 5.6 Sol and Fable 5 posted Pass@1 scores of 43.1% and 41.2%. Two points apart. A single run costs about $1.43 on the first and $9.05 on the second.

That gap is the whole decision, and almost nobody frames it that way. The leaderboards rank by score, so a two-point lead reads as a win, and the price column sits somewhere below the fold where it stops affecting anything.

Read the score as a floor, not a ranking

The first thing those two numbers tell you has nothing to do with which model won.

Both of the top-tier models pass fewer than half the tasks. On real React projects — not toy exercises — the best available reviewer is wrong more often than it is right. Any workflow you build on top of that has to assume a human reads the output, because a tool at 43.1% is a filter, not an approver.

Once you accept that, ranking by two percentage points stops making sense. You are not choosing between a reliable reviewer and a slightly less reliable one. You are choosing between two roughly equivalent filters, and the thing that actually differs between them is what they cost you to run.

I would go further: on a metric where both contenders sit in the low forties, a two-point spread is inside the range where I would not trust the ordering to hold on my codebase. Your React project is not their React project.

The practical consequence is about where you put the human. A 43.1% filter is useful in front of a reviewer and useless in place of one. Wire it so that everything it flags gets looked at and nothing it stays silent about gets waved through, and it earns its keep immediately. Wire it as an approval gate and you have built a machine that ships more than half of what it should have caught, quietly, with a green tick on it.

The cost column is the one that moves

$1.43 against $9.05, per run, on the same benchmark.

Run a reviewer on every pull request and that difference stops being a rounding error by the second week. It compounds in the direction people least expect, too — the cheaper model is the one you can afford to run twice on the same diff, and running a 43% filter twice catches more than running it once.

Token efficiency is the other half of the same story. Sol operates in an ultra-mode with internal multi-agents, which improves task completion accuracy and token efficiency; in agentic coding tasks it is 54% more token-efficient than comparable models. Efficiency of that kind matters most on exactly the work where cost normally spirals — long-running, multi-step tasks on a large codebase, where the context keeps growing and every step pays for it again.

Open Code Review, an open-source tool, pushes this further than any of the frontier models do. Its benchmarks on 200 real PRs and 50 open-source repositories showed higher accuracy and F1 scores than generic agents like Claude Code, while consuming roughly 1/9 of the tokens. That is not a marginal saving. A ninth of the tokens changes which workflows are financially possible at all.

I am a little cautious about vendor-run benchmarks in general, and 200 PRs is not an enormous sample. But the direction is consistent with the architectural claim underneath it: a tool built for one job spends fewer tokens than a general agent talked into doing that job.

There is a second-order effect here that the token count alone does not show. Cheap review changes when you review. At frontier prices you run the reviewer once, at the end, on the finished branch — which is the worst possible moment, because that is when the cost of acting on the feedback is highest. At a ninth of the price you can run it on every commit, and feedback that arrives while you still remember writing the code is worth several times feedback that arrives three days later. The savings show up on the invoice. The actual value shows up in the size of the diffs you end up rewriting.

Specialised beats general, for a structural reason

A general coding agent asked to review a diff has to work out what reviewing means, hold the whole repository context, and decide its own steps. A purpose-built reviewer skips all three.

Agent Skills makes the same bet in a different shape — structured workflows and parallel review mechanisms improve code quality and reduce the likelihood of skipping steps. The gain is not intelligence. It is that a fixed workflow cannot forget step four.

This is why I think the cost comparison above understates the case. The $1.43 and the $9.05 are both frontier models doing a job they were not specifically built for.

Keep the model layer replaceable

If you are building any of this into your own workflow, the architectural advice worth taking is to design the coding agent as a hybrid architecture with a replaceable model-calling layer.

Combine local inference with cloud planning, and optimise task execution through a test-feedback loop. The point is avoiding model binding — and given that the price gap between two models with equivalent scores is currently what it is, being able to swap the model behind your reviewer without rewriting your reviewer is worth the extra interface.

Prices move. Scores move. The two numbers this article is built on will both be stale within a quarter. What will not go stale is the position of that swap point in your architecture.

Concretely, that means the thing your pipeline calls should be your own function, not a vendor SDK sprinkled through eleven files. One place that takes a diff and returns findings, one place that decides which model answers, and every prompt that encodes a particular model’s quirks kept on that side of the line. It is maybe an afternoon of extra work at the start. It is the difference between switching models and rewriting the reviewer.

The tools that are not models

A few things worth knowing about, because they solve problems that no amount of model selection touches.

agent-device lets agents perform operations through CLI commands — opening apps, reading the current page, clicking controls, inputting text, swiping. It is aimed at smoke verification and exploratory testing directly with Codex or Claude Code, which catches a class of problem that reading the diff never will.

stagewise lets front-end AI agents read the current page state directly, improving the accuracy and efficiency of front-end bug fixes. Anyone who has watched an agent reason about a rendering bug from source alone will recognise why that matters.

no-mistakes offers a balance between AI automation and human control through a configurable nine-step pipeline with manual decision points, which is designed around the limits of AI in intent judgment and quality assessment. Those are the two things a 43.1% score is telling you the model cannot do.

Seshport addresses information loss and tone changes when developers switch between AI programming tools, lowering the cost of switching. That problem is downstream of the replaceable-model-layer advice: switching is only cheap if your context survives the switch.

What I would actually do

Start from cost, not from the leaderboard, because the leaderboard is telling you the two candidates are equivalent and the price sheet is telling you they are not.

Run a specialised reviewer rather than a general agent where one exists for your stack — the 1/9 token figure is the largest single number in this whole comparison, and it is the one people skip.

Keep the model behind a swappable interface, so the next time a two-point score difference comes with a six-fold price difference, changing your mind costs you an afternoon.

And read the output like a filter, not a verdict. At 43.1%, that is what it is.

Also readable on Telegraph.

Read next

All 30 write-ups

The 15 figures in this piece — each with the sentence it came from — are in the figures table, alongside 291 more, as JSON and CSV.

Part of ai-coding-field-notes — field notes on AI coding agents.

Want a figure that is not in here yet? Say which metric, which provider, which unit in the open thread — replies get turned into rows. Got a better number? Open an issue — corrections and counter-data are the point. If this collection saved you an afternoon, a star helps other people find it; the data is CC BY and does not require starring.