← All benchmarks·Source: Epoch AI ↗

Aider Polyglot leaderboard

Aider Polyglot leaderboard: multi-language code-editing pass rates for each LLM, plotted against API price to reveal the best value coding models. Current leader: GPT-5 at 88 (% pass).

Cost vs quality

Each dot is a model. The dashed line is the value frontier — top-left is the best trade-off (high score, low cost).

Not enough priced models on this benchmark yet to plot cost vs quality. The leaderboard below shows all scores.

Full leaderboard

#ModelScore% passOutput $/1MReleased
1
OpenAIGPT-5
88—Aug 2025
2
OpenAIo3-pro
84.9—Jun 2025
3
GoogleGemini 2.5 Pro (Jun 2025)
83.1—Jun 2025
4
OpenAIo3
81.3—Dec 2024
5
Grok 4
79.6—Sep 2025
6
GoogleGemini 2.5 Pro (May 2025)
76.9—May 2025
7
DeepSeekDeepSeek-V3.2OPEN
74.2—Dec 2025
8
DeepSeekDeepSeek-V3.2-ExpOPEN
74.2—Dec 2025
9
GoogleGemini 2.5 Pro (Mar 2025)
72.9—Mar 2025
10
AnthropicClaude Opus 4
72—May 2025
11
OpenAIo4-mini
72—Apr 2025
12
DeepSeekDeepSeek-R1 (May 2025)OPEN
71.4—May 2025
13
AnthropicClaude 3.7 Sonnet
64.9—Feb 2025
14
OpenAIo1
61.7—Dec 2024
15
AnthropicClaude Sonnet 4
61.3—May 2025
16
OpenAIo3-mini
60.4—Jan 2025
17
QwenQwen3-235B-A22BOPEN
59.6—Apr 2025
18
QwenQwen3-235B-A22B-Instruct (Jul 2025)OPEN
59.6—Jul 2025
19
Moonshot AIKimi K2 (Jul 2025)OPEN
59.1—Nov 2025
20
DeepSeekDeepSeek-R1OPEN
56.9—Jan 2025
21
GoogleGemini 2.5 Flash (May 2025)
55.1—May 2025
22
DeepSeekDeepSeek-V3 (Mar 2025)OPEN
55.1—Mar 2025
23
Grok 3
53.3—Feb 2025
24
OpenAIGPT-4.1
52.4—Apr 2025
25
AnthropicClaude 3.5 Sonnet (October 2024)
51.6—Oct 2024
26
Grok-3 mini
49.3—Feb 2025
27
DeepSeekDeepSeek-V3OPEN
48.4—Dec 2024
28
GoogleGemini 2.5 Flash (Apr 2025)
47.1—Apr 2025
29
OpenAIGPT-4.5
44.9—Feb 2025
30
OpenAIgpt-oss-120bOPEN
41.8—Aug 2025
31
QwenQwen3-32BOPEN
40—Apr 2025
32
GoogleGemini 2.0 Flash (Dec 2024)
38.2—Dec 2024
33
GoogleGemini 2.0 Pro
35.6—Dec 2024
34
OpenAIo1-mini
32.9—Sep 2024
35
OpenAIGPT-4.1 mini
32.4—Apr 2025
36
AnthropicClaude 3.5 Haiku
28—Oct 2024
37
OpenAIGPT-4o (Aug 2024)
23.1—Aug 2024
38
QwenQwen2.5-Max
21.8—Jan 2025
39
QwQ-32BOPEN
20.9—Mar 2025
40
GoogleGemini 2.0 Flash Thinking (Jan 2025)
18.2—Jan 2025
41
OpenAIGPT-4o (Nov 2024)
18.2—Nov 2024
42
MetaLlama 4 MaverickOPEN
15.6—Apr 2025
43
OpenAIGPT-4.1 nano
8.9—Apr 2025
44
GoogleGemma 3 27BOPEN
4.9—Mar 2025
45
OpenAIGPT-4o mini
3.6—Jul 2024

Scores from Epoch AI, updated —. Cost is joined from our daily API pricing where a model matches; models we don't price still appear with their score. Always verify on the benchmark's own site before relying on a number.

What Aider Polyglot measures

The Aider Polyglot benchmark tests something coding assistants do all day: editing existing code and returning changes in a precise, machine-applicable format. It uses a set of the hardest Exercism programming exercises across multiple languages — including C++, Go, Java, JavaScript, Python and Rust — so it rewards models that are strong beyond just Python.

The score is the percentage of exercises solved, where a solution must both be correct and be delivered as a properly-formatted edit that applies cleanly. That format discipline matters: a model that writes good code but can’t reliably produce a valid diff will score lower — which mirrors real coding-assistant reliability.

How to read this page

Models are ranked by pass rate. As with SWE-bench, the cost-vs-quality chart uses each model’s output price per 1M tokens as the cost axis, since Aider’s public data doesn’t include a uniform per-task cost. The value frontier highlights models that combine a high pass rate with a low generation price.

Limitations

Scores are sensitive to the edit format a model is asked to use and can shift with Aider version changes. The tracked model set also tends to lag the very newest releases, so the freshest frontier models may not appear yet. Read it alongside LiveBench and SWE-bench rather than on its own.

Frequently asked questions

What does the Aider Polyglot score represent?
The percentage of hard, multi-language Exercism coding exercises a model solves, where the answer must be correct and returned as a correctly-formatted edit that applies cleanly. It measures real code-editing reliability, not just raw code generation.
Which languages does it cover?
A spread of languages including C++, Go, Java, JavaScript, Python and Rust, which is why it is called “polyglot” — it rewards models that are strong beyond Python alone.
Why might a capable model score lower here?
Because the benchmark also grades the edit format: a model that writes correct code but fails to produce a valid, cleanly-applicable diff loses points. This intentionally reflects how coding assistants are used in practice.