The most useful number in Cognition's SWE-2 launch is not the 50.0% FrontierCode score. It is the behavior behind that score: SWE-2 medium reportedly reaches its first real edit after a median of 18 steps, while SWE-1.7 takes 48. Cognition says the newer model also costs 81% less on average on the same 100-task evaluation.

That is a better story than another leaderboard victory. A coding agent that starts editing sooner can spend less time crawling a repository, burn fewer credits, and leave less debris for a human to review. But the result does not mean "always pick medium." SWE-2 has three effort levels, and Cognition's own data says high and max still have a job on difficult work.

Cognition's SWE-2 coding model release visual

The useful signal is turn count

Cognition reports four headline results for SWE-2: 50.0% on FrontierCode 1.1 Main, 73.0% on DeepSWE 1.1, 92.8% on Terminal-Bench 2.1, and 27.3% on Terminal-Bench 4. The last number is the awkward one. SWE-2 is close to the top on the first three evaluations, but it trails Fable 5.1 at 55.8% and GPT-6 Astra at 57.9% on Terminal-Bench 4.

That spread is a warning against treating one score as a universal ranking. Terminal-Bench 2.1 and Terminal-Bench 4 are not interchangeable snapshots of "coding intelligence." The tasks, harnesses, and model settings differ. A model can look excellent in one terminal environment and less convincing in another.

FrontierCode is also a vendor-built evaluation. Its methodology is more serious than a prompt-and-vote demo: maintainers write the tasks, each task has reviewer-defined criteria, and Cognition says more than 20 developers spent over 40 hours per task. The benchmark asks whether a change is mergeable, not merely whether the tests pass. Runs that consult solution-bearing web pages are zeroed.

That makes FrontierCode useful evidence, but it does not make it independent evidence. Cognition evaluates outside models in the harness associated with each model, then reports the best result across reasoning settings. That is enough to compare the numbers as a product decision signal. It is not enough to claim that SWE-2 will win on your monorepo, your test suite, or your team's review standard.

The turn count is still meaningful. SWE-2 medium's median first edit at 18 steps versus 48 for SWE-1.7 describes a concrete workflow change. The agent is exploring less before committing to a path. If the path is right, that saves time. If the path is wrong, it can also make the wrong edit earlier. Faster action is only a win when verification catches mistakes.

Cognition says SWE-2 medium scores higher than SWE-1.7 while costing 81% less on average in its FrontierCode runs. It attributes the change to better judgment about which files matter, stronger end-to-end test writing, and fewer redundant reads. Those are exactly the behaviors that lower the cost of an agentic task. They are also the behaviors that need a local test before anyone rewrites a team's default model policy.

A routing rule for real repositories

Use SWE-2 medium for bounded changes with a clear failure signal. Examples include a bug with a reproducible test, a small API change with an existing contract, or a refactor where the affected package is obvious. Medium should be your first test because the reported savings come from doing less unnecessary exploration, not from making the model think forever.

Use high when the repository has several plausible implementation paths, when the change crosses package boundaries, or when the agent must infer behavior that the tests do not spell out. Use max when a task combines uncertain requirements, a long dependency chain, and expensive failure. The extra effort is easier to justify when a bad first patch would create a large review or rollback bill.

Do not select an effort level by task size alone. A 20-line change in an authentication boundary can need more reasoning than a 2,000-line mechanical migration. Route by uncertainty, verification cost, and the price of a wrong edit.

A simple trial can measure whether SWE-2 deserves a place in your stack. Take 20 recent tasks and label each one by uncertainty before running the agent. Use the same repository snapshot, tools, timeout, and test command. Record first-edit step, total turns, elapsed time, credit usage, test result, reviewer changes, and whether the final patch was mergeable. Do not compare a SWE-2 medium run against a high-effort run from another model and call the result fair.

Start with medium on the low-uncertainty group. Escalate to high when the agent misses a requirement, loops on a failing test, or needs to inspect unrelated subsystems. Reserve max for tasks where a second pass is cheaper than a failed handoff. This gives you a routing policy instead of a single winner.

The availability detail matters too. Cognition says SWE-2 is rolling out through Devin Desktop, CLI, Web, and Fusion. The current Devin model documentation still describes SWE-1.7 as the latest listed in-house model and says that quota and extra usage depend on the selected model. That mismatch means readers should check the model selector in their own account before planning around SWE-2. A launch post is not the same thing as a stable rate card or a confirmed seat-level rollout.

The same caution applies to cost. Cognition reports relative savings, not a universal dollar price in the launch text. Your bill depends on the product surface, plan, effort setting, and how many tool calls the task needs. An agent that costs 81% less per benchmark task can still be more expensive in practice if it needs repeated retries on your codebase.

There is another practical limit: SWE-2 is not an open local model just because its base is Kimi K3. Cognition says SWE-2 is post-trained from the 2.8 trillion parameter Kimi K3 model, but the release describes deployment inside Devin products. That lineage is interesting for researchers. It is not a self-hosting instruction for developers with a spare workstation.

The release's training method explains why the effort settings matter. Cognition says it trains all reasoning levels in one reinforcement learning run and applies a cost penalty tuned to the slope of the base model's cost-performance curve. In plain English, the system is optimized to offer several points on the tradeoff, rather than one maximum-effort model with a cheaper label attached.

That is the part worth watching. Agent vendors have spent years reporting a single score and leaving users to discover the bill later. SWE-2 makes cost, turns, and effort part of the product behavior. The claim still needs independent replication, but it points at the metric teams should collect: successful mergeable work per dollar and per human review minute.

For now, the decision is narrow. If you already use Devin, try SWE-2 medium on bounded, testable changes and measure the same task metrics Cognition highlights. Move complex work to high or max only when uncertainty justifies it. If you need a public API, a local weight file, or a guaranteed price, this release does not provide those things. The benchmark is promising. The routing experiment is what tells you whether it matters.

Sources