Field notes on AI coding agents: what they cost, where they break, and what shipped. Figures without a traceable source were cut.
Written with AI assistance. Figures without a traceable source were cut before publishing.
A specialized code review agent beat Claude Code on accuracy across 200 real pull requests and 50 open-source repositories while burning about one-ninth the tokens. That ratio is the whole argument in miniature: the general tool is not worse, it is just paying for generality you may not need. Meanwhile ChatGPT Work saves non-technical staff up to 70% of their time on cross-application tasks, and those same people would get nothing out of an IDE.
So the question is not which tool is best. Here is where hosted tools actually stop being enough, what the switch to an IDE-style tool costs you, and the two mistakes that cost more than picking the wrong tool in the first place.
Lovable will get you a working prototype faster than Cursor will. That is not a knock on it. Hosted tools skip the setup, the file tree, the local environment — you describe the thing and it exists.
The wall is not features. It is the moment your project stops fitting in one conversation.
Everything a hosted tool knows about your work lives in the session. Once the codebase outgrows what fits in context, the tool starts guessing at files it can no longer see, and it guesses confidently. IDE-style tools like Cursor and Claude Code read from disk instead, which is a duller capability than it sounds and the reason they hold up on month three.
Watch the token bill for the same shape. Cloud conversations are cheap. Local project work costs more. Business automation delivery costs more again, and the jump is not proportional to how much code you wrote: it tracks how much context has to be re-read every time. I’d treat a sharply rising bill as the real migration signal, more than any feature comparison.
That progression is worth spelling out, because the three stages fail differently. In the conversation stage you are paying for one file at a time and the cost per useful change stays roughly flat, which is why hosted tools feel so cheap early. In the local project stage the model has to re-read enough surrounding code to stay consistent, so the cost per change starts climbing with the size of the codebase rather than the size of the edit. By the time you are delivering business automation, most of what you pay for is the model reconstructing what it already knew twenty minutes ago, and no amount of prompt tuning fixes that — only tooling that reads from disk does.
The Open Code Review benchmark is the clean version of this argument. Across those 200 pull requests and 50 repositories, the specialized agent was not smarter than Claude Code in any general sense; it just did not have to carry the general case. One-ninth the tokens is what narrowing the problem buys you, and it is the same trade you make when you move a maturing project from a hosted tool onto files.
Nobody writes this part down, so here it is plainly: you lose a week.
Not to learning the interface. To learning that the tool no longer holds your project in its head for you. Hosted tools train a habit of describing what you want and accepting what comes back. IDE-style tools want you to say which files matter, and if you skip that they will happily read the wrong ones.
The second cost is context loss at the boundary. Switching between tools mid-project means the conversation history stays behind, and you end up re-explaining decisions you already made. Tools like seshport exist to carry that history across platforms, which tells you the problem is common enough to have a market.
To be fair, the cost is one-directional. Once you are working from files, moving between IDE-style tools is cheap — they are all reading the same disk.
Match the tool to the person, not the project. This sounds obvious and almost nobody does it.
Sales and finance people running cross-application tasks should be on something like ChatGPT Work. They need the 70% time saving, and they need it without a repository. Handing them Claude Code would be handing them a cockpit.
I suspect this is where most tool decisions actually go wrong inside small companies: one developer evaluates the options, picks correctly for developers, and then the same choice gets rolled out to five people whose work has no files in it at all. The 70% figure is real for the finance analyst pulling numbers across four applications, and it is close to meaningless for the person maintaining a service, because their bottleneck was never the typing.
Developers shipping something they will still be maintaining next quarter should be in an IDE-style tool from the start, even during the prototype phase. The prototype is where the shape of the project gets decided, and that is exactly when you want it on disk where you can read it.
Model choice matters more than tool choice for the actual output. A high-end model on a mediocre tool beats the reverse, because the expensive failure is not a clunky interface — it is rework. Cheap models produce code that looks fine and quietly does not work, and you find out on the fourth file.
Specialized tools are worth a look for anything you do repeatedly. That code review agent hitting higher accuracy at one-ninth the tokens is not magic; it is a narrower problem with a narrower prompt. If you review a lot of pull requests, the math is hard to argue with.
Over-granting permissions is the expensive one. An agent with full system access will eventually run a destructive command against something you did not expect it to touch, and it will do it with no more hesitation than it shows renaming a variable. Give it the narrowest scope the work needs. Keep credentials in .env files rather than anywhere the model can read them casually.
Honestly, the .env habit is the cheapest thing on this list and the one most often skipped, probably because it feels like security theater when you are working alone on a prototype nobody else will ever see. Then the prototype becomes the product, someone else clones it, and the key that was pasted into a config file three months ago is now in a repository with four contributors and no rotation policy.
The point is not that the tool is malicious. It is that it has no model of what is irreplaceable.
Neglecting context management is the quiet one. Left alone, an agent reads whatever is nearby and produces code that is locally reasonable and globally wrong. The fix is boring: break the work into small steps and verify each one before moving on. Iterative development is not a productivity technique here, it is the only way to catch a wrong turn while it is still cheap.
Using something like stagewise for throwaway modifications helps for the same reason — it makes the trial cost low enough that you actually run the trial.
Both mistakes share a shape. They are what happens when you treat the tool as a collaborator who understands the stakes rather than as something extremely fast that has never seen your project before.
Start hosted if you are still deciding what to build. Move to files when the project stops fitting in one conversation. And whichever you pick, decide the permission scope before the first run, not after the first incident — that is the one choice on this list you cannot make retroactively.
Also readable on Telegraph.
Read next
Part of ai-coding-field-notes — field notes on AI coding agents. Found something wrong, or shipped something similar? Open an issue — corrections are the point.