Qwen2.5-Coder Release Notes
Qwen has announced the release of Qwen2.5-Coder, a next-generation series of open-source coding models designed to serve as agile coding partners. The series is available in 1.5B and 7B sizes, with a 32B version scheduled for future release.
Technical Foundation and Training
Qwen2.5-Coder is built upon the Qwen2.5 base and trained on a massive dataset of 5.5 trillion tokens. This training corpus includes source code, synthetic data, and text-code grounding data to enhance specialized coding capabilities. To ensure the models remain versatile for real-world applications like Code Agents, the training also incorporated additional data focused on mathematics and general capabilities.
Base Model Capabilities
The Qwen2.5-Coder base models support a context window of up to 128K tokens and cover 92 programming languages. These models demonstrate significant improvements in several core areas:
- Code Tasks: The models excel in code generation, multi-programming code generation, code completion, and code repair.
- Competitive Performance: The 7B version of Qwen2.5-Coder outperforms larger models, specifically DeepSeek-Coder-V2-Lite and CodeStral-22B, in various code-related evaluation tasks.
- General Proficiency: The models retain the general ability performance of Qwen2.5, as evidenced by evaluations on MMLU and ARC, while remaining competitive in math benchmarks like GSM8K and Math.
Qwen2.5-Coder-Instruct Enhancements
Qwen2.5-Coder-Instruct is the instruction-tuned version of the base model, offering improved task performance and generalization across benchmarks.
Multi-programming and Reasoning
Using McEval to cover over 40 programming languages, Qwen2.5-Coder-Instruct has proven to be an expert in both popular and niche languages. Furthermore, evaluations using CRUXEval indicate that the model excels in code reasoning. The developers noted a correlation between improved code reasoning and a better ability to follow complex instructions.
Mathematics and General Intelligence
Qwen2.5-Coder-Instruct demonstrates strong performance as a "science student," bridging the gap between mathematical foundations and coding tools. In head-to-head comparisons with DeepSeek-Coder-V2-Lite-Instruct, the Qwen2.5-Coder-7B-Instruct model showed superior results across several benchmarks:
| Benchmark | Qwen2.5-Coder-7B-Instruct | DeepSeek-Coder-V2-Lite-Instruct |
|---|---|---|
| Math | 66.8 | 61.0 |
| GaoKao2023en | 60.5 | 56.1 |
| OlympiadBench | 29.8 | 26.4 |
| CollegeMath | 43.5 | 39.8 |
| AIME24 | 10.0 | 6.7 |
General capability benchmarks also favor the 7B-Instruct model, with higher scores in MMLU (68.7 vs 60.6), IFEval (58.6 vs 38.6), and GPQA (35.6 vs 27.6).
Licensing and Availability
Qwen2.5-Coder is released under the Apache 2.0 license to encourage widespread application in code intelligence. The 1.5B and 7B models are currently available, with a 32B version intended to challenge proprietary models coming soon.