Qwen2.5 Release Notes: New Foundation, Coder, and Math Models

Qwen has announced the release of Qwen2.5, a massive open-source update featuring a diverse lineup of general-purpose language models, specialized models for coding (Qwen2.5-Coder) and mathematics (Qwen2.5-Math), and an enhanced Qwen2-VL-72B vision-language model.

Qwen2.5 General Language Models

Qwen2.5 introduces a range of dense, decoder-only models in sizes 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B. These models are pretrained on a large-scale dataset of up to 18 trillion tokens, resulting in significant knowledge gains and improved capabilities over Qwen2.

Key Technical Capabilities

  • Knowledge and Reasoning: The models show improved performance in general knowledge (MMLU: 85+), coding (HumanEval 85+), and mathematics (MATH 80+).
  • Context and Generation: Qwen2.5 supports a context window of up to 128K tokens and can generate up to 8K tokens.
  • Structured Data: There are significant improvements in understanding structured data (such as tables) and generating structured outputs, specifically in JSON format.
  • Instruction Following: The models are more resilient to diverse system prompts, which improves their utility for role-playing and chatbot condition-setting.
  • Multilingual Support: The models support over 29 languages, including English, Chinese, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, and Arabic.

Performance Benchmarks

  • Qwen2.5-72B: This flagship open-source model competes with Llama-3.1-70B and Mistral-Large-V2. The base model of Qwen2.5-72B is reported to reach top-tier performance even against larger models like Llama-3-405B.
  • Mid-Sized Models (14B and 32B): Qwen2.5-14B and Qwen2.5-32B are reintroduced to provide a balance between size and capability, outperforming baseline models like Phi-3.5-MoE-Instruct and Gemma2-27B-IT.
  • Small Language Models (SLMs): Qwen2.5-3B demonstrates high knowledge density, achieving competitive results (MMLU scores above 65) despite its small parameter count.
  • API-based Models: Qwen-Plus is competitive with Llama-3.1-405B and significantly outcompetes DeepSeek-V2.5, though it still trails GPT-4o and Claude-3.5-Sonnet in some areas. Qwen-Turbo provides a cost-effective, rapid service.

Specialized Expert Models

Qwen has released two specialized model families designed for high-performance technical tasks.

Qwen2.5-Coder

Designed for debugging, code suggestions, and answering programming questions, Qwen2.5-Coder was trained on 5.5 trillion tokens of code-related data. It is available in 1.5B, 7B, and 32B (upcoming) sizes. The 7B-Instruct version is noted to outperform many larger language models across various programming languages.

Qwen2.5-Math

Available in 1.5B, 7B, and 72B sizes, Qwen2.5-Math is pretrained on a larger scale of math-related data, including synthetic data generated by Qwen2-Math. It supports both Chinese and English and utilizes reasoning methods including:

  • Chain-of-Thought (CoT)
  • Program-of-Thought (PoT)
  • Tool-Integrated Reasoning (TIR)

The Qwen2.5-Math-72B-Instruct model is reported to surpass both Qwen2-Math-72B-Instruct and GPT-4o in general performance.

Deployment and Tool Integration

Qwen2.5 models are compatible with major industry frameworks for deployment and inference:

  • Hugging Face: Supported via the Transformers library.
  • vLLM: Supports OpenAI API compatible services and built-in tool calling (requires vLLM >= 0.6) using a template inspired by Nous Hermes.
  • Ollama: Supports tool calling via its OpenAI-compatible service.
  • Tool Calling Templates: The chat templates include tool calling support for Hugging Face transformers, while maintaining backward compatibility with the Qwen2 template and Qwen-Agent.

Licensing and Availability

Except for the 3B and 72B variants, all open-source models in the Qwen2.5 release are licensed under Apache 2.0.

Sources