GPQA Diamond leaderboard
GPQA Diamond leaderboard: graduate-level, Google-proof science accuracy for each LLM, plotted against API price to compare deep reasoning ability and cost. Current leader: Gemini 4 Argon at 95.2 (% accuracy).
Cost vs quality
Each dot is a model. The dashed line is the value frontier — top-left is the best trade-off (high score, low cost).
Full leaderboard
OPEN
OPEN
OPENOPENOPENOPEN
OPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPEN
OPENOPENOPENOPENOPENOPENOPENOPENOPEN
OPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENOPENScores from Epoch AI, updated —. Cost is joined from our daily API pricing where a model matches; models we don't price still appear with their score. Always verify on the benchmark's own site before relying on a number.
What GPQA Diamond measures
GPQA Diamond is the 198-question, highest-confidence subset of the 448-question GPQA (Graduate-Level Google-Proof Q&A) benchmark, introduced in 2023 by David Rein, Samuel Bowman, and colleagues at NYU, Cohere, and Anthropic. Each item is a four-option multiple-choice science question written and validated by PhD-level experts in biology, chemistry, or physics. A question earns a place in Diamond only if both expert validators answered it correctly but a majority of skilled non-expert reviewers — given unrestricted web access and 30+ minutes per question — still got it wrong, which is where the "Google-proof" name comes from.
Scoring is raw accuracy: the share of the 198 questions answered with the correct letter choice. Models are usually evaluated with zero-shot chain-of-thought prompting, reasoning through the problem in text before committing to a final answer. Human reference points anchor the scale: random 4-way guessing scores 25%, skilled non-expert web searchers score around 34%, and PhD-holding domain experts score roughly 65–70%.
How to read this page
This benchmark's cost-vs-quality chart uses each model's output price per 1M tokens as a stand-in for a true per-question dollar cost, which GPQA Diamond doesn't publish. That's a reasonable proxy here because scoring runs on chain-of-thought responses: the graded output is the model's full multi-step reasoning trace, not just a single letter, while the input side is only a few sentences of question text plus four answer options. Reasoning-heavy models can burn thousands of output tokens working through one physics or chemistry problem, so output pricing — not input pricing — is where the real cost of running this benchmark shows up.
Limitations
GPQA Diamond's biggest structural limit is its size: only 198 questions means each one is worth roughly half a percentage point, so 1–2 point gaps between models can sit inside the noise. The benchmark's own lead author has acknowledged the Diamond set likely contains some mistakes despite its two-expert-validator screening, and an independent 2025 audit estimated about 8% of Diamond's questions are flawed or ambiguous — putting the practical accuracy ceiling closer to 92% than 100%. Frontier models have converged tightly around 83% accuracy, a saturation pattern that raises the question of whether further gains reflect real reasoning improvement or fitting the dataset's specific distractor quirks. As a static, publicly released dataset (questions and answers have been on GitHub since November 2023), it also carries the standard contamination risk of any fixed benchmark. Finally, its four-way multiple-choice format leaves a 25% guessing floor, and its scope is narrow by design — biology, chemistry, and physics recall-and-reasoning only.