Qwen3 Embedding and Reranker Release
Qwen has released the Qwen3 Embedding series, a set of proprietary models based on the Qwen3 foundation model designed for state-of-the-art text embedding, retrieval, and reranking across more than 100 languages. These models are open-sourced under the Apache 2.0 license and are available on Hugging Face and ModelScope.
Model Capabilities and Performance
The Qwen3 Embedding series achieves state-of-the-art performance in multilingual text understanding and retrieval. As of June 5, 2025, the 8B embedding model ranks first on the MTEB multilingual leaderboard with a score of 70.58.
Reranking Benchmarks
The reranking models demonstrate significant improvements in search relevance across several benchmarks. Performance metrics for the reranker series are as follows:
| Model | MTEB-R | CMTEB-R | MMTEB-R | MLDR | MTEB-Code | FollowIR |
|---|---|---|---|---|---|---|
| Qwen3-Reranker-0.6B | 65.80 | 71.31 | 66.36 | 67.28 | 73.42 | 5.41 |
| Qwen3-Reranker-4B | 69.76 | 75.94 | 72.74 | 69.97 | 81.20 | 14.84 |
| Qwen3-Reranker-8B | 69.02 | 77.45 | 72.94 | 70.19 | 81.22 | 8.05 |
Note: All scores are based on the top-100 candidates retrieved by the Qwen3-Embedding-0.6B model.
Model Specifications
The series provides a flexible range of model sizes (0.6B to 8B) to balance efficiency and effectiveness. Both embedding and reranking models support a sequence length of 32K and are instruction-aware, allowing users to customize input instructions for specific tasks or languages.
Text Embedding Models
| Model | Size | Layers | Embedding Dimension | MRL Support |
|---|---|---|---|---|
| Qwen3-Embedding-0.6B | 0.6B | 28 | 1024 | Yes |
| Qwen3-Embedding-4B | 4B | 36 | 2560 | Yes |
| Qwen3-Embedding-8B | 8B | 36 | 4096 | Yes |
Text Reranking Models
| Model | Size | Layers | Sequence Length | Instruction Aware | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | Qwen3-Reranker-0.6B | 0.6B | 28 | 32K | Yes | | Qwen3-Reranker-4B | 4B | 36 | 32K | Yes | | Qwen3-Reranker-8B | 8B | 36 | 32K | Yes |
Architecture and Training Methodology
The Qwen3 Embedding series utilizes dual-encoder and cross-encoder architectures based on the Qwen3 foundation model, refined via LoRA fine-tuning to preserve the base model's text understanding capabilities.
Architectural Differences
- Embedding Model: Employs a dual-encoder architecture. It processes a single text segment and extracts the semantic representation using the hidden state vector of the final
[EOS]token. - Reranking Model: Employs a cross-encoder structure. It takes text pairs (e.g., a query and a candidate document) as input and outputs a relevance score.
Training Pipeline
- Embedding Models: Follow a three-stage training paradigm: contrastive pre-training with weakly supervised data, supervised training with high-quality labeled data, and a final merging strategy to integrate multiple candidate models.
- Reranking Models: Directly utilize high-quality labeled data for supervised training to maximize efficiency.
To generate large-scale weakly supervised data for the embedding models, Qwen developed a multi-task adaptable prompt system that leverages the Qwen3 foundation model's generation capabilities to create text pairs tailored to specific languages and task types.
Future Roadmap
Qwen plans to continue optimizing the foundation model to improve deployment performance and training efficiency. Additionally, the lab intends to expand the system into a multimodal representation system to enable cross-modal semantic understanding.