TokenCostEstimate bill →
← All benchmarks·Source: Epoch AI ↗

MMLU leaderboard

Current leader: GPT-4o (Nov 2024) at 84.13 (% accuracy).

Cost vs quality

Each dot is a model. The dashed line is the value frontier — top-left is the best trade-off (high score, low cost).

Not enough priced models on this benchmark yet to plot cost vs quality. The leaderboard below shows all scores.

Full leaderboard

#ModelScore% accuracyOutput $/1MReleased
1
GPT-4o (Nov 2024)
84.13—Nov 2024
2
Claude 3.5 Sonnet (October 2024)
83.07—Oct 2024
3
DeepSeek-V3OPEN
82.93—Dec 2024
4
Gemini 1.5 Pro (Sept 2024)
82.53—Sep 2024
5
Claude 3.5 Sonnet
82—Jun 2024
6
GPT-4 (Mar 2023)
81.87—Mar 2023
7
Llama 3.3 70BOPEN
81.73—Dec 2024
8
Gemini 1.5 Pro (May 2024)
81.2—May 2024
9
Qwen2.5-72BOPEN
80.4—Sep 2024
10
Phi-4OPEN
79.73—Apr 2025
11
Claude 3 Opus
79.47—Mar 2024
12
Llama 3.1-405BOPEN
79.33—Jul 2024
13
GPT-4o (Aug 2024)
79.07—Aug 2024
14
GPT-4o (May 2024)
78.93—May 2024
15
Qwen2-72BOPEN
76.53—Jun 2024
16
GPT-4 (Jun 2023)
76.53—Jun 2023
17
Amazon Nova Pro
76—Dec 2024
18
GPT-4o mini
75.73—Jul 2024
19
GPT-4 Turbo (Apr 2024)
75.07—Apr 2024
20
Llama 3.2 90BOPEN
73.73—Sep 2024
21
Llama 3.1-70BOPEN
73.47—Jul 2024
22
Mistral Large 2 (Jul 2024)
73.33—Jul 2024
23
Gemini 2.0 Flash (Dec 2024)
72.93—Dec 2024
24
Llama 3-70BOPEN
72.4—Apr 2024
25
Qwen2.5-Coder-32BOPEN
72.13—Nov 2024
26
Claude 2
71.33—Jul 2023
27
DeepSeek-V2 (MoE-236B, May 2024)
71.2—May 2024
28
phi-3-medium 14BOPEN
70.67—Apr 2024
29
Gemini 1.5 Flash (May 2024)
70.53—May 2024
30
Mixtral 8x22BOPEN
70.4—Apr 2024
31
Yi-34BOPEN
68.4—Nov 2023
32
Claude 3 Sonnet
67.87—Mar 2024
33
Gemma 2 27BOPEN
67.6—Jun 2024
34
phi-3-small 7.4BOPEN
67.6—Apr 2024
35
Qwen2.5-Coder-14B
66.93—Sep 2024
36
Claude 3.5 Haiku
65.73—Oct 2024
37
Gemini 1.5 Flash (Sep 2024)
65.2—May 2024
38
Claude 3 Haiku
65.07—Mar 2024
39
Claude 2.1
64.67—Nov 2023
40
Claude Instant
64.53—Aug 2023
41
Gemma 2 9BOPEN
62.8—Jun 2024
42
GPT-3.5 Turbo (Nov 2023)
61.87—Jun 2023
43
Mixtral 8x7BOPEN
60.8—Dec 2023
44
Falcon-180BOPEN
60.8—Sep 2023
45
Gemini 1.0 Pro
60—Dec 2023
46
Llama 2-70BOPEN
59.87—Jul 2023
47
GPT-3.5 Turbo (Jun 2023)
58.53—Jun 2023
48
Llama 3-8BOPEN
58.4—Apr 2024
49
Mistral Large
58.4—Feb 2024
50
phi-3-mini 3.8BOPEN
58.4—Apr 2024
51
Stable Beluga 2OPEN
58.13—Jul 2023
52
Yi-9B
57.87—Mar 2024
53
Qwen2.5-Coder (7B)OPEN
57.33—Sep 2024
54
GPT-3.5 Turbo (Jan 2024)
56.4—Jun 2023
55
Qwen-14BOPEN
55.07—Sep 2023
56
Gemma 7BOPEN
54.8—Feb 2024
57
StarCoder 2 15BOPEN
52.13—Feb 2024
58
Yi 6BOPEN
52—Nov 2023
59
LLaMA-65BOPEN
51.2—Feb 2023
60
Llama 2-34B
50.13—Jul 2023
61
Mistral 7B v0.1
50—Oct 2023
62
DeepSeek-Coder-V2-Lite-Base
47.33—Jun 2024
63
Baichuan2-13BOPEN
45.6—Sep 2023
64
LLaMA-33BOPEN
44.93—Feb 2023
65
Nemotron-4 15B
44.93—Feb 2024
66
Phi-2OPEN
44.53—Dec 2023
67
Falcon 2 11BOPEN
44.53—May 2024
68
Falcon-40BOPEN
42.53—Mar 2023
69
Llama 3.1-8BOPEN
41.47—Jul 2024
70
Llama 2-13BOPEN
40.8—Jul 2023
71
Baichuan 2-7BOPEN
38.88—Sep 2023
72
Qwen2.5-Coder (1.5B)OPEN
38.13—Sep 2024
73
internlm-7b
34.67—Jul 2023
74
INTELLECT-1
33.19—Nov 2024
75
MPT-30BOPEN
30.53—Jun 2023
76
chatglm2-6b
30.53—Jun 2023
77
LLaMA-13BOPEN
30.27—Feb 2023
78
Llama 2-7BOPEN
27.73—Jul 2023
79
Qwen-7BOPEN
26.67—Sep 2023
80
Gemma 2BOPEN
23.07—Feb 2024
81
Baichuan1-7BOPEN
23.07—Jun 2023
82
Qwen2.5-Coder-0.5B
22.67—Sep 2024
83
CodeQwen1.5-7B
20.67—Apr 2024
84
DeepSeek Coder 33BOPEN
19.2—Jan 2024
85
StarCoder 2 7BOPEN
18.4—Feb 2024
86
Phi-1.5OPEN
16.8—Sep 2023
87
StarCoder 2 3BOPEN
15.47—Feb 2024
88
DeepSeek Coder 6.7BOPEN
15.2—Jan 2024
89
XGen-7BOPEN
15.07—Sep 2023
90
LLaMA-7BOPEN
14.13—Feb 2023
91
Falcon-7BOPEN
13.33—Apr 2023
92
MPT-7BOPEN
7.73—May 2023
93
open_llama_7b
6.53—Jun 2023
94
Qwen-1_8B
4.27—Nov 2023
95
RedPajama-INCITE-7B-BaseOPEN
1.73—Jun 2023
96
Dolly 2.0-12bOPEN
1.6—Apr 2023
97
Cerebras-GPT-13BOPEN
1.6—Apr 2023
98
DeepSeek Coder 1.3BOPEN
1.07—Jan 2024

Scores from Epoch AI, updated —. Cost is joined from our daily API pricing where a model matches; models we don't price still appear with their score. Always verify on the benchmark's own site before relying on a number.