The Token Cost War: Why Price per Million Tokens Now Decides the AI Market

Written with AI assistance. Figures without a traceable source were cut before publishing.

The same piece is posted as a thread on GitHub — that copy has a reply box under it, and this one does not.

The competition among model vendors used to be argued in benchmark scores, and it is now being argued in cost per million tokens — Indian enterprises are adopting Chinese models at input prices as low as $0.19 per million tokens against $5 to $12 for comparable American offerings, which is not a discount so much as a different category of product. I think that single spread explains more about the last six months than any leaderboard does. What changed is not that models got cheaper in the abstract, but that the unit everyone negotiates over moved from raw compute to the effective token production efficiency of a cluster. Below is what that shift looks like at each layer, from silicon to the invoice a solo developer actually pays.

The Metric Moved From Chips to Clusters

Customers stopped asking about single-card performance.

The question a large-model company now brings to a domestic AI chip vendor is narrower and much harder: running inference for a specific model across a thousand, five thousand, or a larger cluster, what is the system’s effective token production efficiency? That framing puts hardware-software co-optimization at the center, because a chip that wins a synthetic benchmark and loses at cluster scale loses the deal. Chinese chip vendors that cannot bring down the generation cost per million tokens do not have a seat at this table at all.

Taichu Yuanqi’s bet is a good illustration of the strategy that follows from it. They emphasize cluster energy efficiency and, deliberately, an independent third-party position — not tied to one model house, not locked to a proprietary stack — and they have completed deep adaptation for more than 40 mainstream models so customers can run them without doing the porting work themselves. I’d argue the independence is the more durable half of that. Compatibility can be matched; being the vendor nobody’s competitor owns is structural.

The other end of the market is buying its way out of the same constraint. Anthropic’s $19 billion data center lease locks in five to eight years of compute, which reframes the competition as a contest over power and floor space rather than architecture. For anyone building on top, the practical reading is that infrastructure cost is now a strategic variable rather than an operational detail.

Aggressive Pricing Arrives at the API Layer

Then the price war reached the API.

Meta’s Muse Spark 1.1 agent model API is priced at $1.25 per million input tokens and $4.25 per million output tokens — roughly 75% and 83% below Anthropic Opus on input and output respectively — and the target is explicit: become the cheap entry point for agent workflows and pull developers into the ecosystem. Aggressive pricing at the agent tier is a different move from aggressive pricing at the chat tier, because agent workloads consume tokens in volumes where an 80% cut changes what is economically possible rather than just what is comfortable.

Cheapest per token is not the same as cheapest per task, though, and this is where I think most cost comparisons go wrong. On ReactBench, one run of GPT-5.6 Sol costs about $1.43 while Fable 5 costs $9.05 for the same work — a gap that no per-token price list would have predicted, because the expensive model is not six times more expensive per token, it is six times less efficient at finishing the job. Evaluate on cost per completed task, in your own environment, with your own prompts.

Efficiency Beats Discounts

The most interesting cost reductions this year did not come from pricing at all.

Alibaba’s open-source Open Code Review is the clearest case. Benchmarked against 200 real pull requests across 50 open-source repositories in 10 programming languages, it scored significantly higher on accuracy and combined F1 than general-purpose agents while consuming roughly one-ninth of the tokens. A ninth. That is a larger cost reduction than any price cut on this page, achieved by a specialized agent doing less redundant reading rather than by anyone lowering a rate.

The same lever appears in aggregation. OmniRoute pools 237 providers into around 1.6 billion free tokens a month and applies RTK plus Caveman compression to squeeze 10,000 tokens down to 1,080, which effectively multiplies whatever free allowance you have by ten. I would want to see what that compression does to output quality on a real workload before treating the ten-times figure as free money, but the direction is right: compression is the one cost lever that does not depend on a vendor’s goodwill.

Self-hosting is the third route, and it trades a subscription for operational work. Open Generative AI packages 200+ models into a self-hosted, uncensored AI video studio, addressing high subscription fees, strict content review and data that lives on someone else’s servers; it has drawn 15K stars and 2.6K forks. Popularity is not the same as production readiness. It is, however, a decent signal of how many people are unhappy with the subscription model.

Small inputs sometimes beat large ones by an absurd margin. Anthropic’s SKILL.md file is 400 tokens of aesthetic guidance and a two-pass working method, and it has been installed over 1.08 million times — a return on 400 tokens that argues taste-shaping is worth more than tool-building, at least in front-end generation.

Where the Money Actually Goes

Enterprise AI spending is out of proportion to what comes back.

Deloitte figures suggest an enterprise with $13 billion in annual revenue may put as much as $700 million into AI, while Jellyfish research finds high token consumption badly out of balance with productivity gains. Those two numbers sit uncomfortably together, and honestly, the gap between them is the whole reason cost efficiency became a boardroom topic rather than an engineering one.

For individual developers the escalation is more legible, and it comes in three thresholds: cloud conversations, then local projects, then automated business delivery. Consumption rises at each step because the role changes — you stop asking questions and start handing over work. Most people who are surprised by a bill crossed a threshold without noticing.

Context is the hidden cost driver.

Token consumption in AI programming assistants comes mostly from repeatedly reading and reprocessing project context, not from the length of the code being generated. That is worth sitting with, because it inverts the intuitive optimization: trimming generated output barely moves the bill, while managing what gets re-read every turn moves it a lot. It also explains why the specialized reviewer above lands at a ninth of the tokens — it is not generating less, it is reading less of the same thing over and over, and I’d argue that is the only cost lever on this list that a single developer can pull today without changing vendors, changing models, or waiting for anyone’s price to drop.

What Commoditization Leaves Behind

Models are becoming shelf goods you can competitively source. Value shifts upward to distribution, identity, data permissions and customer relationships — the parts of a stack that cannot be swapped by changing a base URL. Microsoft’s own framing treats models as parts to be bid on, which is roughly the definition of a commodity.

For small and medium enterprises the same commoditization is straightforwardly good news, since capability that was priced out of reach two years ago is now inside a normal software budget. For anyone selling a thin wrapper over someone else’s model, it is the opposite, and the squeeze is already visible.

What I would take from all of this is narrower than the headlines suggest: stop comparing rate cards and start measuring cost per finished task in your own workload. The vendor with the lowest published price is not reliably the vendor with the lowest invoice.

Also readable on Telegraph.

Read next

All 49 write-ups

The 15 figures in this piece — each with the sentence it came from — are in the figures table, alongside 393 more, as JSON and CSV.

Topics: Artificial Intelligence · AI

Part of llm-api-pricing — field notes on AI coding agents.

Did this save you an afternoon? A star on the repository is the whole ask — it is what puts these in front of the next person looking; the data is CC BY and does not require starring.

One thing this piece could not settle: What do you actually pay per million input tokens on the model you use most? Reply with a number. The reply box is on the thread copy of this piece.

Want a figure that is not in here yet? Say which metric, which provider, which unit — in one line. One required field, and the page you came from is already filled in. Got a better number? Open an issue — that form knows which write-up you came from too; corrections and counter-data are the point.