The Cheapest LLM for Coding: SWE-bench Score per Dollar

Every few weeks a new model tops the SWE-bench Verified leaderboard, and every few weeks a wave of teams switch their coding agent to it — often without checking what that top spot costs. A model that resolves 3 percentage points more issues than the runner-up, at 4× the price, is usually the wrong trade for a production coding assistant. Here's how to think about it properly.

What SWE-bench Verified actually measures

SWE-bench Verified is a set of real, human-validated GitHub issues; a model passes a task only if its patch makes that repository's own test suite go green. There's no partial credit and no self-grading — it's one of the closest proxies we have to "can this model fix real bugs in a real codebase." That makes the score genuinely meaningful, unlike some benchmarks that reward answer formatting as much as reasoning.

But the score alone doesn't tell you what it costs to get that resolved issue — and for a coding agent that runs continuously across a team's workflow, cost per resolved issue compounds fast.

The value-per-dollar concept

Instead of asking "which model scores highest," ask "which model gives me the most resolved issues per dollar spent." Plot every model's SWE-bench score against its price (we do this automatically on the SWE-bench leaderboard, with cost on one axis and score on the other) and a pattern always emerges: a handful of models sit on the value frontier — no other model available is simultaneously cheaper and higher-scoring. Everything below or to the right of that frontier is strictly dominated: you could switch to a frontier model and get equal-or-better results for equal-or-less money.

The model at the very top of the score axis is almost never on the frontier for long — it typically got there by spending more per task (more reasoning tokens, more tool calls, higher per-token price), and a model just a few points lower often costs a fraction as much.

Why this matters more for coding than for chat

Coding agents are usage-heavy in a way chat interfaces aren't:

• Agentic loops make many calls per task — planning, tool calls, retries, verification steps — so a per-token price difference multiplies across a single resolved issue many times over, not once.
• Output is the expensive side. Code generation is output-token-heavy (see Input vs Output Tokens), and coding agents often produce long diffs, explanations, and multiple attempts — exactly the tokens priced highest.
• Teams run these constantly, not occasionally, so a marginal per-task cost difference becomes a real monthly line item at scale, not a rounding error.

How to actually pick a model

1. Open the SWE-bench Verified leaderboard and look at the cost-vs-quality chart, not just the ranked table. Identify which models sit on the value frontier for your budget range.
2. Set a quality floor first. Decide the minimum resolve rate you need for your use case (e.g., "must be within a few points of the top score"), then find the cheapest model that clears that floor — don't start from price and work up, or you'll end up under-provisioned on capability.
3. Cross-check against a second signal. Aider Polyglot measures multi-language code-editing reliability rather than repository-level bug fixing — a model can rank differently across the two, and agreement between them is a stronger signal than either alone.
4. Price your actual agent loop, not just one call. If your agent averages several model calls per resolved task, multiply accordingly before comparing to a simpler single-call estimate — the calculator can model steady-state volume once you know your average calls-per-task.
5. Re-check periodically. New models enter the frontier often; a model that was the best value three months ago may now be dominated by a newer, cheaper release.

The takeaway

"Best coding model" and "best value coding model" are different questions, and for most teams running an agent continuously, the second one is what actually shows up on the invoice. Use the cost-vs-quality view on benchmarks to find your value frontier, confirm with a second benchmark, and re-rank against your own budget rather than chasing whichever model is on top of a single leaderboard this week.