Claude Code, Codex, and Cursor are quietly making procurement decisions inside codebases. They are choosing databases, email providers, payment systems, storage platforms, and deployment tools, then writing the integration. That changes the question from "Which coding agent writes better code?" to "Which agent gets to pick a dependency, and when do I stop it?"
A new Armature study gives that question an uncomfortable answer. Across 16,893 runs, the three agents selected the same tool in only 42% of comparison cells. In other words, the model or harness you use can change a dependency decision before a human has looked at the tradeoff. That is not a reason to crown a winner. It is a reason to treat agent recommendations as proposals that need a policy.
The timing matters. Vercel says coding agents now initiate over 30% of deployments on its platform, up 1,000% from six months earlier. When an agent chooses a provider while it is implementing a feature, that choice can move directly into a running system. The dependency decision is no longer a research step that happens before coding. It is part of the coding loop.
What the study actually measured
Armature retained 5,292 valid sessions from the full 16,893 runs. The experiment used 75 repositories across 10 languages, 1,163 prompt variations, and four personas ranging from vague “make this work” requests to enterprise-style requirements. Three agents, Claude Code, Codex, and Cursor, worked in ephemeral sandboxes. A simulated human asked for a recommendation first and then approved the top choice, which is closer to a real workflow than asking an agent to silently install the first library it finds.
That design is useful, but it is not a neutral benchmark. Armature sells growth services to developer-tool companies. The authors disclose that the research is also about influencing which products coding agents pick. I would not throw the results away. I would read them as behavioral evidence with a commercial incentive attached, then inspect the public traces before turning any percentage into a procurement rule.
The clearest split is how the agents gather evidence. Codex searched the web in 94% of sessions, usually with operators such as site:. Cursor used the web in about two thirds of sessions. Claude Code searched in about 30%, but when it did browse, it visited roughly three times as many pages as Codex. Claude Code also built an in-house solution in 19% of sessions, compared with 10% for Codex and Cursor.
Those are workflow differences, not intelligence scores. A web-heavy agent may discover a current pricing limit that another agent misses. A prior-heavy agent may move faster when the repository already contains strong conventions. An agent that prefers building in-house may avoid a bad vendor choice, or it may spend two days maintaining a service that a managed product would have handled.
Repository context changed the answer even more than the agent identity. In four versions of the same email task, across four languages, Resend won on TypeScript with 55 of 89 runs, SendGrid won on Python with 22 of 24, Postmark won on Go with 20 of 24, and Azure Communication Services won on Java with 22 of 23. The prompt was similar. The codebase was not.
That is the part I would pin above every autonomous coding setup: recommendations are conditional on the repository. A provider that looks sensible in a TypeScript project can lose immediately when the agent sees Java conventions, existing Azure credentials, or a different deployment target.
The decision matrix
| Situation | Better default | Why | Human approval |
|---|---|---|---|
| You need current pricing, limits, or integration docs | Codex, or Cursor with browsing enabled | Codex searched in 94% of sessions and tends to narrow searches with site: operators |
Required before committing to a paid service |
| The repository has mature conventions and internal wrappers | Claude Code | Its lower browsing rate can preserve local context, and it built in-house in 19% of sessions | Required when it proposes replacing an existing pattern |
| You are choosing a provider for a language-specific project | Run two agents against the same clean prompt | The study saw different winners for TypeScript, Python, Go, and Java | Required, compare evidence rather than votes |
| The task involves compliance, data residency, or customer records | Any agent only as a research assistant | A persuasive recommendation cannot establish legal or contractual fit | Mandatory human review |
| You are choosing a small, reversible utility library | The agent with the shortest verified path | The cost of a wrong choice is limited if the dependency is easy to remove | Review the lockfile and license |
| The agent wants to build a service in-house | Ask for a managed alternative and a maintenance estimate | Claude Code built in-house in 19% of sessions, almost twice the 10% rate of the other two | Mandatory for production systems |
The matrix is deliberately boring. That is the point. I would not set a policy that says "always use Codex for research" or "Claude Code understands codebases best." I would set conditions around the evidence the agent must show.
For a new dependency, require the agent to return the rejected alternatives, the current price or free-tier limit, the data location, the license, the expected monthly volume, and the exact files it changed. If it cannot answer one of those questions, it can prepare a branch but cannot merge it. For a payment provider or anything handling personal data, require a human to approve even when all three agents agree.
There is a practical way to test this without adopting a giant evaluation harness. Take five real backlog tasks that involve an external service. Give the same sanitized repository state and the same constraints to two agents. Record what each agent searched, which options it rejected, what it installed, and what changed in the diff. The goal is not to produce a leaderboard. The goal is to find the categories where your own codebase causes disagreement.
If you already have more than one coding tool, keep a local record of usage and task outcomes instead of trusting subscription dashboards. Our local AI coding usage dashboard review covers that visibility problem. Knowing which agent made a choice is useful; knowing whether the choice survived review is much more useful.
Where the evidence breaks
The 42% agreement number is not a universal disagreement rate. Armature calls it agreement across comparison cells, and the retained set is only the portion judged valid and publishable. The experiment also used a Gemini 3.7 Flash orchestrator as the simulated human and another Gemini instance as the judge. That keeps the setup repeatable, but it adds a second model's assumptions to every run.
The study also does not prove that the minority choice was wrong. In fact, disagreement can be healthy. Stripe won nine out of ten payment sessions, yet specialized providers won in some EU-regulated cases. A model that picks Paddle for a European tax requirement may be doing better than one that blindly follows the majority. Popularity is a clue, not a decision criterion.
Mention counts are another trap. PayPal appeared 139 times in payment conversations and was never selected. Adyen appeared 175 times and was selected only three times. LangChain was mentioned 194 times and picked four times. Supabase was the most mentioned database but was often beaten by Neon. The agent can name a product because it is familiar, then reject it because the repository needs something narrower. Humans do this too, although usually with fewer logs.
The commercial disclosure matters most when interpreting the recommendation behavior of vendors. If a company is measuring how to get products picked by agents, its results can still reveal what agent-mediated procurement looks like, but the raw traces and methodology deserve more attention than the headline. I would use this study to design a review process, not to optimize a vendor's landing page or declare one coding tool the market winner.
The strongest conclusion is simple: the codebase is part of the prompt, whether you write that prompt down or not. Language, framework, existing credentials, lockfiles, deployment target, and even a product page's wording can flip the recommendation. Let an agent search. Let it propose. Let it implement behind a branch. Do not let a 20-minute conversation quietly become a three-year dependency.
Sources
- Armature coding-agent tool-choice study: methodology, 16,893 runs, 5,292 retained sessions, agent search behavior, and repository-context results
- Vercel Agentic Infrastructure report: official deployment data showing coding agents initiated over 30% of deployments
- Hacker News discussion of the Armature study: community discussion and the authors' conflict-of-interest disclosure