Qwen2-Math Release Notes

Qwen has introduced Qwen2-Math, a series of specialized large language models designed to solve complex arithmetic and mathematical problems. The flagship model, Qwen2-Math-72B-Instruct, demonstrates state-of-the-art performance, outperforming proprietary models such as GPT-4o, Claude-3.5-Sonnet, Gemini-1.5-Pro, and Llama-3.1-405B on key mathematical benchmarks.

Model Architecture and Training

Qwen2-Math models are built upon the Qwen2 foundation and are available in three sizes: 1.5B, 7B, and 72B. The development process consists of two primary stages:

Base Model Pretraining

The base models are initialized from Qwen2 and pretrained on a specialized Mathematics-specific Corpus. This dataset includes:

  • High-quality mathematical web texts and books.
  • Code and exam questions.
  • Mathematical pre-training data synthesized by Qwen2.

Instruction Tuning

To create the Instruct versions, Qwen developed a math-specific reward model based on Qwen2-Math-72B. The training pipeline utilizes:

  • Rejection Sampling: Combining dense reward signals with binary correctness signals to construct Supervised Fine-Tuning (SFT) data.
  • Reinforcement Learning: Implementing Group Relative Policy Optimization (GRPO) following the SFT phase.

Performance and Evaluation

Qwen2-Math was evaluated across a wide array of English and Chinese benchmarks to test both general and competition-level mathematics.

General Math Benchmarks

The models were tested on GSM8K, Math, and MMLU-STEM. For the Instruct models, performance was measured using Greedy, Maj@8, and RM@8 (Reward Model) settings. The results indicate that RM@8 outperforms Maj@8, particularly in the 1.5B and 7B models, validating the effectiveness of the math reward model.

Competition-Level Mathematics

Qwen2-Math-Instruct was tested on highly challenging datasets, including:

  • English: OlympiadBench, CollegeMath, AIME 2024, and AMC 2023.
  • Chinese: GaoKao (2024), CN Middle School 24, and CMATH.

In complex competition evaluations like AIME 2024 and AMC 2023, the model demonstrated strong performance across Greedy, Maj@64, RM@64, and RM@256 settings.

Data Decontamination

To ensure the integrity of the benchmarks, Qwen applied rigorous decontamination methods to both pretraining and post-training datasets:

  • Exact Match: Removing identical samples found in test sets.
  • 13-gram Deduplication: Removing samples where the longest common sequence ratio exceeds 0.6.
  • Targeted Filtering: Removing contaminated samples from datasets including GSM8K, MATH, Aqua, SAT Math, OlympiadBench, College Math, AIME24, and AMC23.

Current Limitations and Roadmap

Qwen2-Math currently primarily supports English. The lab has announced plans to release bilingual models (English and Chinese) and is developing multilingual models to further expand accessibility and capability.

Sources