Qwen2.5 LLM series release

TL;DR

Qwen released the Qwen2.5 series – a family of decoder‑only LLMs from 0.5B to 72B parameters that are up to twice as capable as the previous Qwen2 models while being fully open‑source for the 3B, 14B and 32B sizes. The upgrades include a larger 18‑trillion‑token pre‑training corpus, major gains in knowledge, coding, mathematics, and alignment, and a context window of up to 128K tokens.

Overview of the Qwen2.5 Release

  • Model lineup – Seven open‑source models (0.5B, 1.5B, 3B, 7B, 14B, 32B, 72B). The 3B, 14B and 32B checkpoints are newly added to meet demand for production‑grade (10‑30B) and mobile‑side (≈3B) workloads.
  • Licensing – All models are Apache‑2.0 except Qwen2.5‑3B (Qwen Research License) and Qwen2.5‑72B (Qwen License).
  • Context & generation – Context length up to 128 K tokens; generation length up to 8 K tokens, enabling long‑form output.
  • Access – In addition to the open‑source checkpoints, Qwen‑Plus and Qwen‑Turbo are offered via Alibaba Cloud Model Studio API.

Technical Improvements Over Qwen2

Upgrade Detail
Dataset size Pre‑training data increased from 7 T tokens to a maximum of 18 T tokens.
Knowledge MMLU scores rise from 70.3 → 74.2 (7B) and 84.2 → 86.1 (72B). Improvements also observed on GPQA, MMLU‑Pro, MMLU‑Redux, ARC‑C.
Coding Introduced Qwen2.5‑Coder. Qwen2.5‑72B‑Instruct scores 55.5 (LiveCodeBench), 75.1 (MultiPL‑E) and 88.2 (MBPP), surpassing Qwen2‑72B‑Instruct (32.2, 69.2, 80.2).
Mathematics Integrated Qwen2‑math technology. MATH benchmark jumps to 75.5 (7B) and 83.1 (72B) from 52.9 / 69.0.
Alignment Arena‑Hard score climbs from 48.1 → 81.2 and MT‑Bench from 9.12 → 9.35 for the 72B instruction model.
Other capabilities Instruction following, long‑text generation (1 K → 8 K tokens), table understanding, JSON generation, and robustness to diverse system prompts are all markedly better.

Model Card Highlights

Model Params Layers Heads (KV) Context Generation License
Qwen2.5‑0.5B 0.49 B 24 14/2 32 K 8 K Apache 2.0
Qwen2.5‑1.5B 1.54 B 28 12/2 32 K 8 K Apache 2.0
Qwen2.5‑3B 3.09 B 36 16/2 32 K 8 K Qwen Research
Qwen2.5‑7B 7.61 B 28 28/4 128 K 8 K Apache 2.0
Qwen2.5‑14B 14.7 B 48 40/8 128 K 8 K Apache 2.0
Qwen2.5‑32B 32.5 B 64 40/8 128 K 8 K Apache 2.0
Qwen2.5‑72B 72.7 B 80 64/8 128 K 8 K Qwen

Benchmark Performance – Base Models

72B Model

Qwen2.5‑72B outperforms peers across the board and reaches scores comparable to Llama‑3‑405B while using only ~1/5 of the parameters. Highlights:

  • MMLU: 86.1 (vs. 84.2 for Qwen2‑72B, 85.2 for Llama‑3‑405B)
  • BBH: 86.3 (vs. 82.4 for Qwen2‑72B)
  • GPQA: 45.9 (vs. 37.4)
  • MATH: 62.1 (vs. 50.9)
  • HumanEval: 59.1 (vs. 64.6 for Qwen2‑72B, but higher than many open models)
  • Multilingual: Best on Multi‑Exam (78.7) and Multi‑Understanding (89.6).

Mid‑size Models

  • Qwen2.5‑14B beats Qwen1.5‑32B and other 14‑30B open models on MMLU (79.7) and BBH (78.2).
  • Qwen2.5‑32B surpasses Qwen2‑57B‑A14B and reaches Llama‑3‑70B‑level performance on many tasks (e.g., MMLU 83.3, GSM8K 92.9, MBPP 84.5).

7B Model

Qwen2.5‑7B improves over Qwen2‑7B despite fewer non‑embedding parameters (6.5 B vs. 6.5 B). Key scores: MMLU 74.2, MATH 49.8, HumanEval 57.9.

Edge‑Side Models (≤3B)

All three models (0.5B, 1.5B, 3B) show consistent gains over their Qwen2 counterparts, especially on coding (e.g., HumanEval 42.1 → 74.4 for 3B) and math (MATH 19.5 → 42.6 for 3B).

Instruction‑Tuned Models

Qwen2.5‑72B‑Instruct

  • Arena‑Hard: 81.2 (vs. 48.1 for Qwen2‑72B‑Instruct)
  • MATH: 83.1 (vs. 69.0)
  • LiveCodeBench: 55.5 (vs. 32.2)
  • MT‑Bench: 9.35 (vs. 9.12)
  • Outperforms Llama‑3.1‑405B‑Instruct on several metrics (e.g., MMLU‑Pro 71.1 vs. 73.3 for Llama‑3.1‑405B‑Instruct).

Qwen‑Turbo, Qwen2.5‑14B‑Instruct, Qwen2.5‑32B‑Instruct

  • Qwen2.5‑32B‑Instruct reaches the highest scores among open models of similar size (e.g., MMLU‑Pro 69.0, GSM8K 95.9, HumanEval 88.4).
  • Qwen‑Turbo (API) delivers competitive results comparable to GPT‑4o‑mini on many benchmarks while remaining open‑source.

7B, 3B, 1.5B, 0.5B Instruction Models

All smaller instruction models show large relative improvements over their base versions, making them viable for resource‑constrained deployments. Notable examples:

  • Qwen2.5‑7B‑Instruct: MATH 75.5, HumanEval 84.8.
  • Qwen2.5‑3B‑Instruct: Coding (HumanEval 74.4) and math (MATH 65.9) comparable to larger proprietary models.
  • Qwen2.5‑1.5B‑Instruct: HumanEval 61.6, MATH 55.2 – a >20‑point jump from Qwen2‑1.5B‑Instruct.

Multilingual Capabilities

The series was evaluated on translated IFEval, language‑specific MMLU variants, extended MGSM8K, and cultural‑nuance benchmark BLEnD. The 72B instruction model achieves the highest multilingual scores among the listed competitors (e.g., 86.98 on multilingual IFEval, 80.56 on Japanese JMMLU). The 14B and 7B instruction models also close the gap with larger proprietary systems on many languages.

New Functionalities Demonstrated

  • JSON generation – The models reliably emit syntactically correct JSON structures, useful for tool‑calling pipelines.
  • Long‑form generation – With an 8 K token generation limit, Qwen2.5 can produce essays, reports, or codebases without truncation.
  • Structured data understanding – Table parsing and multi‑column reasoning are markedly better, as shown in the demo suite.

Implications for the LLM Landscape

  • Open‑source competitiveness – By releasing 14B and 32B checkpoints that outperform many proprietary models, Qwen narrows the gap between open and closed ecosystems.
  • Production‑ready middle tier – The 10‑30B range is now covered by fully open models, lowering entry barriers for enterprises seeking self‑hosted LLMs.
  • Edge deployment – The 3B model delivers strong coding and math abilities on a footprint suitable for mobile or embedded devices.
  • Alignment progress – Substantial gains in Arena‑Hard and MT‑Bench indicate that open‑source instruction tuning can achieve human‑preference alignment comparable to leading closed models.

Where to Get the Models

All numbers and tables are reproduced verbatim from the Qwen2.5 announcement; no additional claims have been added.

Sources