Databricks just published the uncomfortable version of AI coding economics: a model with cheaper tokens can still cost more to finish the same task. In its internal benchmark, GLM 5.2 averaged $1.28 per coding task, Claude Opus 4.8 averaged $1.94, and Sonnet 5 averaged $2.09. Sonnet was the cheaper model by token price. It still lost on the bill and on completion quality.

Databricks cost and savings chart for AI coding workloads

That is the useful part of the new Databricks cost report. The headline is not that one model won. The lesson is that teams should stop buying tokens and start measuring completed work. If you are deciding whether to swap a default model, build a router, or add a meta-harness, cost per passing task is the number that makes the decision less stupid.

The cost-per-task trap

Anthropic's current pricing makes the trap easy to see. Opus 4.8 is listed at $5 per million input tokens and $25 per million output tokens. Sonnet 5's introductory price through August 31, 2026 is $2 input and $10 output. On a spreadsheet, Sonnet is the obvious bargain.

Databricks found that its coding benchmark did not behave like that spreadsheet. Sonnet scored 81% on task completion against 87% for Opus 4.8, and consumed 1.9 times more tokens. Its average task cost ended at $2.09, slightly above Opus at $1.94. The model price was lower. The work required to reach an acceptable patch was not.

This is the distinction most AI cost dashboards hide. A request can be cheap while the task is expensive. A coding agent reads files, calls tools, retries tests, repairs its own mistakes, and repeats context across turns. The first prompt is usually a tiny part of the input bill. A model that needs more turns can erase its discount before the pull request is ready.

Databricks also saw the same problem at the expensive end. It reports a cost regression when comparing Opus 5.0 with Opus 4.8. Stripe reportedly found that Opus 4.7 did not provide a meaningful quality gain over Opus 4.6 while costing more, so it did not make the newer model an internal default. Newer is not a cost strategy.

The report's figures are internal and directional, not a universal leaderboard. That caveat matters. Databricks used tasks from a multi-million-line codebase and its own harnesses. Your repository, tests, context window, and review policy will change the result. But the measurement pattern transfers cleanly.

Start with the unit that your team actually cares about: a merged, reviewed, passing change. For each task, log the model, harness, input tokens, output tokens, retries, wall-clock time, test result, and final spend. Keep the task set fixed while you change one variable. Eight representative tasks are more useful than a generic benchmark if those eight expose your real failure modes.

The existing eight-task evaluation probe is a good starting point. Add a cost column and a hard pass rule. A task that produces code quickly but needs a human rewrite is not a win.

A routing decision that survives contact with your codebase

There are three reasonable moves, and they solve different problems.

If most of your tasks are routine, change the default model first. Configuration edits, small bug fixes, test updates, and mechanical refactors should not automatically consume your most expensive reasoning model. Databricks found GLM 5.2 statistically tied with Opus 4.8 on its benchmark while averaging $1.28 per task. That is a strong case for testing an open model as a daily driver, but not for routing every design question to it.

If task shapes vary wildly, use request-level routing. A stateful proxy can send each inference request to the lowest-cost model that meets a quality target. The hard part is context. A router that ignores cache state and repeated context can choose the cheaper model and still pay too much. Route with task cost, quality, availability, and cache behavior in the same policy.

If the harness is the waste, routing models alone will disappoint. Databricks says the same model and thinking effort cost more than twice as much per task in different harnesses while quality stayed similar. Its simpler Pi harness sent about three times less context per turn than Claude Code or Codex in the comparison. The model was not the only variable. The wrapper was feeding it a larger bill.

This is why a meta-harness can be useful. Omnigent, Databricks' open source project, wraps agents such as Claude Code, Codex, and Pi behind a common interface. In practice, the value is less about another chat window and more about keeping model choice, permissions, logging, and cost policies outside a single vendor's client. That buys you the ability to test a different model without asking every developer to rebuild their habits.

A router also needs an escalation rule. Start the routine task on the cheaper model. Escalate when the task touches multiple packages, changes an API, fails a test twice, or asks for design work rather than implementation. The exact trigger should come from your failure logs, not from a generic complexity classifier that sounds smart in a demo.

Databricks reports that its Smart Router reduced average task cost by more than 30% while roughly matching the quality of the most expensive model in its working set. It also reports up to 90% savings in some scenarios across the stack. Those are useful targets, not promises. Treat them as a reason to run your own replay, not as a reason to buy a gateway.

A small team should probably not build the whole platform on day one. First run a two-week replay over merged pull requests. Compare your current default against one cheaper model and one premium model. Then test a simple rule based on task labels. If the cheap model passes routine work and the premium model wins on design-heavy work, you have evidence for routing. If the results are noisy, keep one default and fix the harness before adding infrastructure.

What to measure before changing the default

Use four buckets: routine edits, bug fixes, multi-file features, and design or migration work. For each bucket, record at least twenty tasks if you can. The important calculation is:

cost per passing task = total model cost + retry cost + review cost divided by passing tasks

You can estimate review cost in engineer minutes. That keeps the model's apparent savings honest when its patch takes a person forty minutes to understand. It also exposes a common failure mode: the cheapest model can produce a larger diff, more retries, and a higher review burden.

Keep separate numbers for first-pass success and eventual success. A model that reaches 90% eventual success after four retries may be worse for a live coding loop than a model that reaches 85% on the first attempt. Latency also matters. A dollar saved is not automatically worth fifteen minutes of blocked developer time.

Then look at context. Databricks says its own harness and cache tuning cut generated tokens and related cost by almost 50% without observed quality degradation. The cheapest improvement may be removing verbose tool output, compacting context more often, or splitting a large task before you touch model selection. The prompt caching and token overhead guide covers that complementary lever.

Do not impose a hard monthly cutoff as your first control. Databricks argues that hard budgets can punish the developers who get the most value from agents, while still failing to explain why spend rose. A better sequence is visibility, a warning, a self-clearing gate, and then downshifting to a cheaper model. Suspension is the emergency brake.

The decision rule is simple. Switch the default when the cheaper model wins cost per passing task in the routine bucket and does not create a review problem. Add routing when the task buckets have different winners and the routing overhead is lower than the observed savings. Keep the premium model when it wins on the work where failure is expensive. Change the harness before changing models when context volume explains most of the gap.

That is the part worth copying from Databricks. Not the product names, and not the idea that GLM 5.2 should be everyone's default. The useful move is to make the benchmark belong to your codebase, then let the numbers decide where a premium model earns its keep.

Sources