FunAudioLLM/CosyVoice
Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.
What it solves
CosyVoice 是一个基于大型语言模型（LLM）的文字转语音（TTS）系统，旨在提供高质量的零样本多语言语音合成。它解决了在“野外”场景下，如何生成自然且在不同语言和方言之间保持说话人相似度和内容一致性的难题。
How it works
系统利用 LLM 生成语音。最新版本（Fun-CosyVoice 3.0）侧重于规模扩展和后训练，以提升韵律自然度和说话人相似度。它支持零样本语音克隆，意味着只需短短的样本即可模仿说话人的声音，无需大量重新训练。系统还包括文字规范化过程，能够处理数字和符号，无需传统前端模块。
Who it’s for
该项目面向需要可扩展、高性能 TTS 系统的开发者和研究者，尤其是需要多语言和跨语言合成，或构建需要低延迟流式输出的生产级音频应用的用户。
Highlights
- Multilingual & Dialect Support: Covers 9 common languages and over 18 Chinese dialects/accents.
- Zero-Shot Voice Cloning: Supports multi-lingual and cross-lingual voice cloning.
- Low Latency: Achieves audio-out streaming latency as low as 150ms.
- Controllability: Supports pronunciation inpainting for Pinyin and English phonemes, and instructions for emotion, speed, and volume.
- Deployment Options: Compatible with vLLM and Nvidia TensorRT-LLM for accelerated inference.