← All benchmarks·Source: Epoch AI

Aider Polyglot leaderboard

Aider Polyglot leaderboard: multi-language code-editing pass rates for each LLM, plotted against API price to reveal the best value coding models. Current leader: GPT-5 at 88 (% pass).

Cost vs quality

Each dot is a model. The dashed line is the value frontier — top-left is the best trade-off (high score, low cost).

Not enough priced models on this benchmark yet to plot cost vs quality. The leaderboard below shows all scores.

Full leaderboard

#ModelScore% passOutput $/1MReleased
1
GPT-5
88Aug 2025
2
o3-pro
84.9Jun 2025
3
Gemini 2.5 Pro (Jun 2025)
83.1Jun 2025
4
o3
81.3Dec 2024
5
Grok 4
79.6Sep 2025
6
Gemini 2.5 Pro (May 2025)
76.9May 2025
7
DeepSeek-V3.2OPEN
74.2Dec 2025
8
DeepSeek-V3.2-ExpOPEN
74.2Dec 2025
9
Gemini 2.5 Pro (Mar 2025)
72.9Mar 2025
10
Claude Opus 4
72May 2025
11
o4-mini
72Apr 2025
12
DeepSeek-R1 (May 2025)OPEN
71.4May 2025
13
Claude 3.7 Sonnet
64.9Feb 2025
14
o1
61.7Dec 2024
15
Claude Sonnet 4
61.3May 2025
16
o3-mini
60.4Jan 2025
17
Qwen3-235B-A22BOPEN
59.6Apr 2025
18
Qwen3-235B-A22B-Instruct (Jul 2025)OPEN
59.6Jul 2025
19
Kimi K2 (Jul 2025)OPEN
59.1Nov 2025
20
DeepSeek-R1OPEN
56.9Jan 2025
21
DeepSeek-V3 (Mar 2025)OPEN
55.1Mar 2025
22
Gemini 2.5 Flash (May 2025)
55.1May 2025
23
Grok 3
53.3Feb 2025
24
GPT-4.1
52.4Apr 2025
25
Claude 3.5 Sonnet (October 2024)
51.6Oct 2024
26
Grok-3 mini
49.3Feb 2025
27
DeepSeek-V3OPEN
48.4Dec 2024
28
Gemini 2.5 Flash (Apr 2025)
47.1Apr 2025
29
GPT-4.5
44.9Feb 2025
30
gpt-oss-120bOPEN
41.8Aug 2025
31
Gemini 2.0 Flash (Dec 2024)
38.2Dec 2024
32
Gemini 2.0 Pro
35.6Dec 2024
33
o1-mini
32.9Sep 2024
34
GPT-4.1 mini
32.4Apr 2025
35
Claude 3.5 Haiku
28Oct 2024
36
GPT-4o (Aug 2024)
23.1Aug 2024
37
Qwen2.5-Max
21.8Jan 2025
38
GPT-4o (Nov 2024)
18.2Nov 2024
39
Gemini 2.0 Flash Thinking (Jan 2025)
18.2Jan 2025
40
Llama 4 MaverickOPEN
15.6Apr 2025
41
GPT-4.1 nano
8.9Apr 2025
42
Gemma 3 27BOPEN
4.9Mar 2025
43
GPT-4o mini
3.6Jul 2024

Scores from Epoch AI, updated . Cost is joined from our daily API pricing where a model matches; models we don't price still appear with their score. Always verify on the benchmark's own site before relying on a number.

What Aider Polyglot measures

The Aider Polyglot benchmark tests something coding assistants do all day: editing existing code and returning changes in a precise, machine-applicable format. It uses a set of the hardest Exercism programming exercises across multiple languages — including C++, Go, Java, JavaScript, Python and Rust — so it rewards models that are strong beyond just Python.

The score is the percentage of exercises solved, where a solution must both be correct and be delivered as a properly-formatted edit that applies cleanly. That format discipline matters: a model that writes good code but can’t reliably produce a valid diff will score lower — which mirrors real coding-assistant reliability.

How to read this page

Models are ranked by pass rate. As with SWE-bench, the cost-vs-quality chart uses each model’s output price per 1M tokens as the cost axis, since Aider’s public data doesn’t include a uniform per-task cost. The value frontier highlights models that combine a high pass rate with a low generation price.

Limitations

Scores are sensitive to the edit format a model is asked to use and can shift with Aider version changes. The tracked model set also tends to lag the very newest releases, so the freshest frontier models may not appear yet. Read it alongside LiveBench and SWE-bench rather than on its own.

Frequently asked questions

What does the Aider Polyglot score represent?
The percentage of hard, multi-language Exercism coding exercises a model solves, where the answer must be correct and returned as a correctly-formatted edit that applies cleanly. It measures real code-editing reliability, not just raw code generation.
Which languages does it cover?
A spread of languages including C++, Go, Java, JavaScript, Python and Rust, which is why it is called “polyglot” — it rewards models that are strong beyond Python alone.
Why might a capable model score lower here?
Because the benchmark also grades the edit format: a model that writes correct code but fails to produce a valid, cleanly-applicable diff loses points. This intentionally reflects how coding assistants are used in practice.