Edge0-AI/Audio8_TTS

What it solves

Audio8_TTS is a compact, multilingual text-to-speech (TTS) system designed for high-quality speech generation and zero-shot voice cloning. It aims to provide a powerful yet portable model that can perform voice cloning without requiring additional training for new speakers, while maintaining a small parameter footprint (0.6B or 0.1B parameters).

How it works

The system uses a DualAR (Dual Autoregressive) architecture. It consists of a "Slow AR" transformer that predicts semantic tokens for each audio frame, and a "Fast AR" transformer that predicts the codec codebooks based on the slow hidden state and previous codebooks. The model bundles its own neural codec for waveform decoding, eliminating the need for a separate model.

For the 0.1B version, the pure-attention slow AR backbone is replaced with a Falcon-H1 hybrid (Mamba 2 SSM + attention) to further reduce size and increase efficiency.

Who it’s for

• Developers looking for a lightweight, portable TTS model for integration into applications.
• Researchers interested in zero-shot voice cloning and efficient AR architectures.
• ** uma l** users who need high-quality speech synthesis in multiple languages (including English, Chinese, Japanese, Korean, and several European languages).

Highlights

• Zero-shot Voice Cloning: Ability to clone a voice using only a short reference audio clip and its transcript.
• Compact Size: Available in 0.6B and 0.1B parameter versions, making it highly efficient.
• Multilingual Support: Optimized for 11 languages, including Cantonese, Chinese, English, French, German, Italian, Japanese, Korean, Polish, and Spanish.
• High Performance: Competitive with much larger SOTA models on benchmarks like Seed-TTS and CV3.
• Deployment Options: Supports CPU ONNX Runtime (INT4) for low-memory deployment and SGLang Omni serving for high-throughput, OpenAI-compatible API serving.