Qwen2.5-Coder Series Release Notes

Qwen has open-sourced the Qwen2.5-Coder series, a family of code-specialized large language models designed to provide state-of-the-art (SOTA) open-source coding capabilities across a wide range of model sizes. The flagship Qwen2.5-Coder-32B-Instruct model matches the coding performance of GPT-4o, establishing a new benchmark for open-source code LLMs.

SOTA Coding Capabilities of Qwen2.5-Coder-32B-Instruct

Qwen2.5-Coder-32B-Instruct is the flagship model of the series, demonstrating competitive performance against GPT-4o across several critical programming dimensions:

Code Generation and Repair

Qwen2.5-Coder-32B-Instruct achieves top performance among open-source models on the EvalPlus, LiveCodeBench, and BigCodeBench benchmarks. In code repair—the ability to fix errors in existing code—the model scored 73.7 on the Aider benchmark, performing comparably to GPT-4o.

Code Reasoning and Multi-Language Support

The model excels in code reasoning, which involves predicting inputs and outputs by learning the process of code execution. It supports over 40 programming languages, scoring 65.9 on the McEval benchmark. This proficiency extends to multi-language code repair, where it ranked first among open-source models with a score of 75.2 on the MdEval benchmark.

Human Preference Alignment

Using an internal "Code Arena" benchmark (similar to Arena Hard) and GPT-4o as the evaluation model, Qwen2.5-Coder-32B-Instruct demonstrated superior alignment with human preferences compared to other open-source models.

Diverse Model Sizes and Scaling

Qwen2.5-Coder is available in six different sizes to accommodate various hardware constraints and developer needs. Each size is provided as both a Base model (for fine-tuning) and an Instruct model (for direct chat interaction).

Model	Parameters	Non-Emb Params	Layers	Heads (KV)	Tie Embedding	Context Length	License
Qwen2.5-Coder-0.5B	0.49B	0.36B	24	14 / 2	Yes	32K	Apache 2.0
Qwen2.5-Coder-1.5B	1.54B	1.31B	28	12 / 2	Yes	32K	Apache 2.0
Qwen2.5-Coder-3B	3.09B	2.77B	36	16 / 2	Yes	32K	Qwen Research
Qwen2.5-Coder-7B	7.61B	6.53B	28	28 / 4	No	128K	Apache 2.0
Qwen2.5-Coder-14B	14.7B	13.1B	48	40 / 8	No	128K	Apache 2.0
Qwen2.5-Coder-32B	32.5B	31.0B	64	40 / 8	No	128K	Apache 2.0

Evaluation on MBPP-3shot (for Base models) and the most recent four months of LiveCodeBench (for Instruct models) confirms a positive correlation between model size and performance, with Qwen2.5-Coder achieving SOTA performance across all available sizes.

Practical Applications and Integration

Qwen has focused on the real-world utility of the series through two primary integration scenarios:

AI Code Assistants (Cursor)

Qwen2.5-Coder is designed to be a powerful open-source alternative for code assistants like Cursor. The 32B model specifically demonstrates SOTA code completion capabilities (Fill-in-the-Middle mode) on five benchmarks: Humaneval-Infilling, CrossCodeEval, CrossCodeLongEval, RepoEval, and SAFIM.

Visual Artifacts (Open WebUI)

Through integration with Open WebUI, Qwen2.5-Coder can be used to create "Artifacts"—visual works such as websites, mini-games, and data charts. Qwen also plans to launch a dedicated code mode on the Tongyi official website to support one-click generation of these visual applications.