Qwen1.5-32B release notes / what's new
Qwen has introduced Qwen1.5-32B and Qwen1.5-32B-Chat to provide a high-performance alternative to larger models that reduces memory consumption, inference latency, and finetuning costs. These models are positioned as a "sweet spot" for users who find 14B models insufficient for complex tasks like agent scenarios and 72B models too resource-intensive.
Technical Architecture and Efficiency
Qwen1.5-32B maintains the same general architecture as the rest of the Qwen1.5 series, with the primary exception of the inclusion of Grouped Query Attention (GQA). The implementation of GQA allows for more efficient model serving and improved inference performance compared to standard attention mechanisms.
Performance Benchmarks
Qwen1.5-32B demonstrates competitive capabilities across base, chat, and multilingual evaluations, often outperforming other models in the 30B parameter range.
Base Model Capabilities
In base capability evaluations, Qwen1.5-32B outperforms Llama2-34B and Mixtral-8x7B in most tasks. While it performs slightly below the Qwen1.5-72B model, it remains highly competitive in reasoning and coding benchmarks:
| Model | MMLU | C-Eval | GSM8K | MATH | HumanEval | MBPP | BBH | CMMLU |
|---|---|---|---|---|---|---|---|---|
| Qwen1.5-32B | 73.4 | 83.5 | 77.4 | 36.1 | 37.2 | 49.4 | 66.8 | 82.3 |
| Qwen1.5-72B | 77.5 | 84.1 | 79.5 | 34.1 | 41.5 | 53.4 | 65.5 | 83.5 |
| Mixtral-8x7B | 70.6 | - | 74.4 | 28.4 | 40.2 | 60.7 | - | - |
| Yi-34B | 76.3 | 81.4 | 67.2 | 14.4 | 23.2 | 41.0 | 54.3 | 83.7 |
| Llama2-34B | 62.6 | - | 42.2 | 6.2 | 22.6 | 33.0 | 44.1 | - |
Chat Model Evaluation
Qwen1.5-32B-Chat utilizes advanced post-training techniques, including RLHF, to enhance conversational abilities. It achieves a score over 8 on MT-Bench, with a relatively small performance gap when compared to the 72B-Chat variant:
| Models | MT-Bench (Avg. Score) | AlpacaEval 2.0 (LC Win Rate) |
|---|---|---|
| Qwen1.5-32B-Chat | 8.30 | 27.49 |
| Qwen1.5-72B-Chat | 8.61 | 36.60 |
Multilingual and Long-Context Performance
Qwen1.5-32B supports 12 languages, including Arabic, Spanish, French, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, and Indonesian. Its average performance across exams, understanding, math, and translation is 56.91, which is higher than Mixtral-8x7B (50.39) and slightly behind Qwen1.5-72B (60.44).
Additionally, the model achieves top-level performance in "Needle in a Haystack" evaluations for context lengths up to 32K tokens.