Qwen3-TTS Release Notes: Open-Source Voice Design, Cloning, and Generation
Qwen has open-sourced the Qwen3-TTS family, a series of speech generation models capable of high-fidelity voice cloning, natural language-based voice design, and precise acoustic control. Available in 0.6B and 1.7B parameter sizes, these models support 10 mainstream languages and various dialects, providing a versatile toolkit for real-time interactive speech applications.
Technical Architecture and Core Innovations
Qwen3-TTS utilizes a discrete multi-codebook language model (LM) architecture to achieve full-information end-to-end speech modeling. This approach eliminates the information bottlenecks and cascading errors typically found in traditional LM+DiT (Diffusion Transformer) schemes.
Qwen3-TTS-Tokenizer-12Hz
The system is powered by the Qwen3-TTS-Tokenizer-12Hz multi-codebook speech encoder. This tokenizer enables efficient acoustic compression and high-dimensional semantic modeling while preserving paralinguistic information and acoustic environmental features. This allows for high-speed, high-fidelity speech reconstruction using a lightweight non-DiT architecture.
Dual-Track Streaming Generation
To meet the demands of real-time interaction, Qwen3-TTS implements a Dual-Track hybrid streaming generation architecture. This allows a single model to support both streaming and non-streaming generation, delivering the first audio packet after processing just a single character. The end-to-end synthesis latency is as low as 97ms.
Model Variants and Capabilities
The Qwen3-TTS family is divided into two primary sizes: the 1.7B model for peak performance and control, and the 0.6B model for efficiency.
| Model | Key Features | Language Support | Streaming | Instruction Control |
|---|---|---|---|---|
| Qwen3-TTS-12Hz-1.7B-VoiceDesign | Voice design via natural language descriptions | 10 Languages | Yes | Yes |
| Qwen3-TTS-12Hz-1.7B-CustomVoice | Style control over target timbres; 9 premium timbres | 10 Languages | Yes | Yes |
| Qwen3-TTS-12Hz-1.7B-Base | 3-second rapid voice clone; base for fine-tuning | 10 Languages | Yes | No |
| Qwen3-TTS-12Hz-0.6B-CustomVoice | 9 premium timbres | 10 Languages | Yes | No |
| Qwen3-TTS-12Hz-0.6B-Base | 3-second rapid voice clone; base for fine-tuning | 10 Languages | Yes | No |
Supported languages include Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, and Italian.
Key Functional Features
Natural Language Voice Design
Users can create entirely new voice identities by providing free-form natural language descriptions. This includes specifying acoustic attributes (pitch, speed, volume), persona descriptions, and background information. The model can adapt tone, rhythm, and emotional expression based on these instructions and the semantics of the text.
Rapid Voice Cloning and Cross-Lingual Generalization
The Base models can perform rapid voice cloning using only a 3-second audio sample. This capability extends to cross-lingual cloning, allowing a speaker's timbre to be maintained across different languages.
Precise Acoustic Control
Qwen3-TTS supports multi-dimensional control over speech output, including:
- Single Attribute Control: Adjusting specific elements like emotion (e.g., "very angry") or pace (e.g., "extremely slow").
- Multi-Attribute Control: Combining gender, pitch, speed, and personality traits into a complex prompt.
- Timbre Reuse: The ability to store and recall specific timbres to generate multi-character, long-form dialogues.
Performance Benchmarks
Voice Design and Control
On the InstructTTS-Eval benchmark, Qwen3-TTS-VoiceDesign outperformed the closed-source MiniMax-Voice-Design model in both instruction-following and generative expressiveness. For voice control tasks, Qwen3-TTS-Instruct achieved a score of 75.4% on InstructTTS-Eval and maintained a Word Error Rate (WER) of 2.34% for single-speaker multilingual generalization.
Voice Cloning
On the Seed-tts-eval benchmark, Qwen3-TTS surpassed MiniMax and SeedTTS in speech stability for Chinese and English cloning. In a 10-language multilingual test set, it achieved an average WER of 1.835% and a speaker similarity of 0.789, outperforming MiniMax and ElevenLabs.
Tokenizer Quality
Evaluated on the LibriSpeech test-clean set, the Qwen-TTS-Tokenizer achieved SOTA results across several metrics:
- PESQ: 3.21 (wideband) and 3.68 (narrowband).
- STOI: 0.96
- UTMOS: 4.16
- Speaker Similarity: 0.95