SWE-bench Verified leaderboard
SWE-bench Verified leaderboard: % of real GitHub issues resolved by each LLM, plotted against API price so you can compare coding ability and cost together. Current leader: Claude Opus 4.7 at 83.47 (% resolved).
Cost vs quality
Each dot is a model. The dashed line is the value frontier — top-left is the best trade-off (high score, low cost).
Full leaderboard
Scores from Epoch AI, updated —. Cost is joined from our daily API pricing where a model matches; models we don't price still appear with their score. Always verify on the benchmark's own site before relying on a number.
What SWE-bench Verified measures
SWE-bench Verified is the closest thing the field has to “can this model actually fix real software.” It’s a set of 500 real GitHub issues from popular open-source Python projects that have been human-validated for being well-specified and solvable — a cleaned-up subset of the original SWE-bench. A model is given the repository and the issue, and must produce a code patch.
The score is the percentage of issues resolved: a task counts only if the model’s patch makes the repository’s own test suite pass. There’s no partial credit and no self-grading — the tests are the judge.
How to read this page
The leaderboard ranks models by percent resolved. Because SWE-bench doesn’t publish a consistent per-task token cost across models, the cost-vs-quality chart uses each model’s output price per 1M tokens as the cost axis — a reasonable proxy, since agentic bug-fixing is output-heavy. Read it as “resolve rate per dollar of generation.”
Limitations
SWE-bench Verified is Python-centric, so it doesn’t directly reflect performance in other languages. Results are also sensitive to the agent scaffold — the harness, retry logic and tools around the model — so the same model can score differently under different setups. And the Verified subset is smaller and cleaner than full SWE-bench, so numbers aren’t comparable across the two variants.