QwQ-32B release notes / what's new

Qwen has introduced QwQ-32B, a 32-billion parameter model that utilizes scaled reinforcement learning (RL) to achieve reasoning capabilities comparable to the 671-billion parameter DeepSeek-R1. This release demonstrates that applying RL to robust foundation models with extensive world knowledge can significantly enhance intelligence and reasoning efficiency.

Scaling Reinforcement Learning for Reasoning

QwQ-32B uses a multi-stage reinforcement learning approach to scale reasoning capabilities beyond traditional pretraining and post-training methods.

Stage 1: Domain-Specific RL

In the initial phase, RL is scaled specifically for mathematics and coding tasks. To ensure accuracy without relying on traditional reward models, Qwen implemented outcome-based rewards using:

  • Accuracy Verifiers: Used for math problems to validate the correctness of final solutions.
  • Code Execution Servers: Used to assess whether generated code successfully passes predefined test cases.

Stage 2: General Capability RL

Following the domain-specific phase, a second stage of RL focuses on general capabilities. This stage utilizes rewards from a general reward model and rule-based verifiers. This brief training phase improves instruction following, human preference alignment, and agent performance without causing a significant drop in math or coding proficiency.

Model Capabilities and Performance

QwQ-32B is designed for complex problem-solving, mathematical reasoning, and coding. It integrates agent-related capabilities, allowing the model to think critically while using tools and adapting its reasoning based on feedback from its environment.

Performance evaluations show that QwQ-32B is competitive with leading models, including:

  • DeepSeek-R1
  • o1-mini
  • DeepSeek-R1-Distilled-Llama-70B
  • DeepSeek-R1-Distilled-Qwen-32B

Deployment and Accessibility

QwQ-32B is an open-weight model released under the Apache 2.0 license. It is available through the following channels:

  • Hugging Face and ModelScope: Open-weight access for local deployment via Transformers.
  • Qwen Chat: Accessible via the web interface.
  • Alibaba Cloud DashScope API: Available for programmatic integration.

Future Research Directions

Qwen intends to further explore the intersection of foundation models and scaled RL to move toward Artificial General Intelligence (AGI). Key areas of future focus include:

  • Inference Time Scaling: Unlocking greater intelligence by scaling computation during the inference process.
  • Long-Horizon Reasoning: Integrating agents with RL to enable the model to handle complex, multi-step reasoning tasks over longer durations.

Sources