ARC-AGI-2 leaderboard
ARC-AGI-2 leaderboard: percent of novel abstract-grid puzzles solved by each LLM, plotted against API price to compare genuine generalization and cost. Current leader: GPT-6 Astra at 95 (% solved).
Cost vs quality
Each dot is a model. The dashed line is the value frontier — top-left is the best trade-off (high score, low cost).
Full leaderboard
OPENOPEN
OPEN
OPENOPEN
OPENOPENOPENOPENOPEN
OPENOPEN
OPENScores from Epoch AI, updated —. Cost is joined from our daily API pricing where a model matches; models we don't price still appear with their score. Always verify on the benchmark's own site before relying on a number.
What ARC-AGI-2 measures
ARC-AGI-2 is the second iteration of the Abstraction and Reasoning Corpus, created by François Chollet (also known for the original ARC-AGI benchmark and for creating Keras) with Mike Knoop, and released by the nonprofit ARC Prize Foundation in March 2025. Each task shows a model 2–4 example pairs of input and output grids (up to 30×30 cells, 10 possible colors) that all share one hidden transformation rule, then asks it to infer that rule and apply it to novel test grids it has never seen. A model gets pass@2 — two attempts per test grid, the same protocol used with human testers — and a task counts solved only on an exact, cell-for-cell match to the ground-truth grid, with no partial credit.
The full corpus holds 1,360 tasks: 1,000 public training tasks plus 360 evaluation tasks split evenly into public, semi-private, and private sets of 120 each, so outside evaluators can score models on held-out tasks without exposing the ones used to guard against memorization. ARC-AGI-2 was purpose-built to defeat the brute-force program-search ensembles that had pushed ARC-AGI-1 past 85% without much genuine generalization, weighting tasks toward symbolic interpretation, compositional reasoning, and contextual rule application that resist combinatorial search.
How to read this page
This page's cost-vs-quality chart plots each model's output price per 1M tokens as the cost axis, which is a reasonable stand-in here because ARC-AGI-2 has no single official dollar-per-task figure of its own: solving a task well typically means a model works through a long internal chain of reasoning (testing candidate transformation rules against the demonstration grids) before committing to a final answer grid, so output/reasoning tokens dominate the token bill far more than the short grid-based input prompt does. That said, published benchmarking has shown "% solved" can hide order-of-magnitude differences in actual dollar cost per task depending on search scaffolding and thinking budget rather than list price alone, so read the output-price proxy as directional, not exact.
Limitations
ARC-AGI-2's maintainers caution that scores below roughly 5% are largely noise rather than signal, and the slice actually used for outside scoring is just 120 tasks per set, so a handful of tasks flipping solved/unsolved can swing a model's percentage by close to a point. Performance is also famously scaffold- and compute-sensitive rather than a clean property of the base model: published 2025 testing found cost per task ranging from under $2 to over $30 for broadly comparable accuracy depending on search strategy and thinking budget, so the same model can post very different "% solved" numbers depending on how much compute and engineering scaffolding surrounds it. There's also a contamination question: despite the held-out private set, researchers have observed frontier models referencing ARC-specific grid and color conventions unprompted, suggesting some exposure via the openly published training and public eval sets may be leaking into pretraining data. Finally, like its predecessor, ARC-AGI-2 tests a narrow slice of intelligence (abstract visual/symbolic grid reasoning) and says nothing directly about language understanding, factual knowledge, or real-world agentic tasks.