FrontierCode leaderboard

What FrontierCode measures

FrontierCode is a coding-agent benchmark from Cognition. Each task pairs a checked-out repository with one real, hard open-source issue, and the agent works on it autonomously in a container. The question it asks is stricter than whether the tests pass: is the patch good enough to merge? Submissions are graded against held-out tests and a rubric written by maintainers that covers behavioral correctness, regression safety, build and style cleanliness, and adherence to project conventions.

Version 1.1 reports two nested subsets. The Main set holds the 100 hardest tasks and Extended holds all 150. Epoch AI charts the Main score, which is each model's rubric score on those 100 tasks at its best-performing reasoning effort, averaged over five runs. If a patch fails any blocker criterion it scores zero, and runs flagged for consulting solution-bearing sources on the internet are zeroed.

How to read this page

Models are run through agent harnesses such as Claude Code, Codex, mini-SWE-agent and Devin, and Epoch keeps each model's harness and best effort in its data export. The cost chart here uses output price per 1M tokens, which is only a rough guide: an agent run is a long loop of reading files, running commands and editing, so total spend depends on how many steps the model takes, how much context it re-reads, and whether caching is available, not just on its list price. A model that costs less per token but takes many more steps can end up more expensive per merged fix.

Limitations

Scores depend on the harness and reasoning effort as well as the model, so two entries are not always a like-for-like comparison. The set is small (100 tasks in Main), the rubric is a single vendor's definition of mergeable, and the results are sourced from Cognition's public leaderboard rather than run independently by Epoch. The leading models are close together, so treat small gaps as ties. FrontierCode replaces SWE-bench Verified on this site: Epoch's own page labels SWE-bench Verified as flawed, and its most recent results here stopped well before the current generation of models.