Field notes on AI coding agents: what they cost, where they break, and what shipped. Figures without a traceable source were cut.
Written with AI assistance. Figures without a traceable source were cut before publishing.
“Choosing the right GPT-5.6 model for your business is more about avoiding cost overruns than just picking the cheapest option.”
The problem isn’t about finding the cheapest model—it’s about finding the one that delivers real value without hidden costs. Luna’s 95% accuracy rate at 1/3 the cost of Terra shows how much you can save by making the right choice. Terra requires 3x more tokens for equivalent tasks, and its 800ms response time makes it far less efficient for time-sensitive applications.
Most businesses assume cheaper models are always better.
Terra’s document processing capabilities might seem cost-effective initially, but Luna’s 95% accuracy on basic QA tasks and faster response times mean fewer errors and rework, whereas Terra’s higher failure rate on complex tasks can lead to time wasted fixing mistakes, and Luna’s superior accuracy and reliability make it a better long-term choice, even though Terra often requires more tokens for similar tasks.
The reality is that using cheaper models often leads to more costly rework.
Choosing cheaper models can lead to hidden costs like rework and platform lock-in, as seen in the shift from hosted tools to IDE-based tools for long-term projects. When working with AI tools, choosing the right model (like Luna for complex tasks) ensures higher accuracy and fewer errors, reducing the need for iterative adjustments that Terra’s cheaper alternative might require. Luna’s architecture supports higher concurrent requests compared to Terra. This scalability advantage, combined with its lower error rate and faster processing, means Luna is more cost-effective overall, even at slightly higher initial costs.
Luna’s focus on accuracy and reliability, which aligns with the principle that AI programming tools should use flagship models to avoid the pitfalls of cheaper alternatives, means that businesses can prioritize long-term efficiency over short-term savings, thereby avoiding the time wasted on fixing errors and rework.
For simple, well-defined tasks, using cheaper models can be cost-effective.
Luna handles basic customer service queries with ease, and while its high accuracy rate for such questions makes it ideal for simple tasks, Terra excels at document summarization tasks, with its strengths lying in more complex scenarios, as shown by its performance in document analysis; Understanding your specific needs is key to choosing the right model for your tasks.
When evaluating cheaper models, it’s important to consider their limitations. While Luna and Terra perform well in their respective domains, they may struggle with more nuanced or ambiguous tasks. For example, Luna’s 95% accuracy rate for basic questions drops when faced with more complex queries, and Terra’s document analysis accuracy can vary depending on document structure and content.
In real-world scenarios, where cheaper models often require additional context or manual intervention to achieve satisfactory results, for instance, Luna’s basic customer service queries may need supplementary information to handle edge cases, and Terra’s document summarization may require human review for critical information, These additional steps can offset the cost savings of using cheaper models.
While cheaper models like Luna and Terra show promise in specific domains, their limitations become apparent when applied to more complex tasks. Luna shows reliable performance in basic question-answering scenarios, which aligns with its strengths in high-concurrency customer service use cases, but real-world applications often require more nuanced understanding. Terra, meanwhile, shows potential in document analysis tasks, as it can extract and generate standard analytical documents for daily business needs, though the variability in results suggests that human oversight remains necessary for critical tasks.
In addition to Luna and Terra, developers also need to make a careful choice between ChatGPT Work and Claude Code. Terminal developers should continue using Claude Code, as the cross-application context collection and multi-step task automatic execution functions provided by ChatGPT Work have limited impact on improving developer efficiency. This is because the performance difference between the two tools becomes clear when faced with actual development tasks, and the capabilities of Claude Code are more in line with the needs of developers.
OpenAI’s strategic move is also worth noting. Through the Sol model and ChatGPT Work strategy, OpenAI has transformed AI Agent capabilities from being a developer tool to a standard for office work. This has reduced enterprise costs and expanded the user base. AI Agent capabilities are now accessible to all, with non-developers and free users enjoying the same benefits. This has greatly lowered the threshold for using AI Agents.
Luna consumes more tokens for equivalent tasks, which shows how quickly context grows in larger projects. You also need to consider integration costs, as they can add up quickly. Open Code Review requires 1/9th the tokens of standard agents and offers a 20% higher accuracy rate.
For example, Luna’s 95% accuracy in basic Q&A is great for customer service, but the error correction time can be a drawback. Terra’s ability to extract all data is ideal for document processing, but its 2x token usage may be a concern for large-scale projects.
The key is finding a balance between cost and performance. While Luna’s high accuracy is useful, Terra’s document extraction capabilities offer cost-effectiveness for large-scale tasks. Open Code Review’s higher accuracy and lower token usage make it an excellent choice for code review.
Beyond the direct costs, indirect expenses like additional training and setup for Miora’s Pro and Max tiers and workflow reconfiguration when switching models should be considered. Independent developers must carefully evaluate these hidden costs to find the most cost-effective solution, which may not always be the cheapest option initially.
The most overlooked aspect is context management. Luna’s higher token consumption for equivalent tasks highlights the rapid growth of context in larger projects. Terra’s ability to maintain full project context and Open Code Review’s deterministic engineering ensure efficiency and consistency, respectively.
Codex++’s multi-API injection capability shows how proper tool selection reduces operational overhead.
“Start with your specific use case.”
Luna’s 95% accuracy rate for basic questions makes it ideal for simple tasks, while Terra’s strength shows up in document analysis and more complex scenarios. For example, Luna can handle customer service queries efficiently, while Terra excels at processing complex financial reports. The choice between these models depends on whether you prioritize speed or depth of analysis.
“Consider your team’s expertise.”
Luna takes less developer time, which shows how much more efficient it is overall, and Terra’s 2x more tokens for equivalent tasks adds up quickly in larger projects. Junior developers benefit most from Luna’s simplified workflow, while senior developers may find Terra’s deeper analysis capabilities more useful for complex debugging tasks.
“Don’t forget about long-term costs.”
Luna’s 95% accuracy rate reduces rework costs, and Terra’s higher token costs add up over time. For startups with limited budgets, Luna’s cost-effective approach may be more sustainable, while established teams with complex needs might justify Terra’s higher upfront investment through reduced long-term maintenance costs.
The model selection process should consider not just immediate needs but also future scalability. Luna’s architecture is optimized for horizontal scaling, making it ideal for businesses expecting rapid growth, while Terra’s vertical scaling capabilities may better suit organizations with predictable, high-volume requirements. Luna’s integration with existing cloud infrastructure reduces costs for teams already invested in particular cloud providers.
Grill-me’s iterative questioning process reduces decision-making friction, helping developers align on requirements more effectively, especially in ambiguous scenarios. By forcing developers to clarify their intentions, Grill-me’s approach minimizes the risk of costly rework due to initial assumptions. This practical method saves useful time in real-world development workflows.
Agent evaluation frameworks must address five critical dimensions: functional correctness (P0), process quality (P1), efficiency and cost (P2), robustness and security (P3), and user experience (P4). The most rigorous evaluations combine automated testing for functional correctness with human-in-the-loop validation for process quality assessments. Token consumption monitoring provides useful insights into operational efficiency, while security testing ensures the agent can handle sensitive data appropriately.
Open Code Review’s 1/9 token consumption advantage over general-purpose agents translates to large cost savings for development teams. The tool’s hybrid architecture combining deterministic engineering with agent capabilities ensures both high accuracy and operational efficiency. This makes it particularly useful for small teams and independent developers who need high-quality code reviews without the high costs associated with more general-purpose solutions.
When evaluating AI programming tools, consider the model’s influence on your team’s workflow. Tools that simplify processes, such as Luna, can make productivity better by reducing developer time largely. On the other hand, tools like Terra, which may demand more resources for similar tasks, are better suited for teams with ample budgets. The decision between these tools should be guided by your team’s unique requirements and constraints.
A mid-sized e-commerce company cut its model spend over 6 months. It switched from Terra to Luna for a large portion of queries, which reduced debugging time and sped up development.
These improvements highlight the importance of model selection and its impact on overall project efficiency.
Also readable on Telegraph.
Read next
Part of ai-coding-field-notes — field notes on AI coding agents. Found something wrong, or shipped something similar? Open an issue — corrections are the point.