Audio8-AI/Audio8_TTS

SOTA-Class TTS at Compact Scale

What it solves

Audio8_TTS is a compact, multilingual text-to-speech (TTS) system designed for high-quality speech generation and zero-shot voice cloning. It addresses the need for efficient, small-parameter models that can mimic a speaker's voice from a short reference clip without requiring additional training for that specific voice.

How it works

The system uses a DualAR (Dual Autoregressive) architecture. A "slow" AR transformer predicts semantic tokens for audio frames, which are then processed by a "fast" AR transformer that predicts the codec codebooks. The model includes its own neural codec for waveform decoding, eliminating the need for a separate model. It is available in two main sizes: a 0.6B parameter preview model and a highly portable 0.1B parameter version that utilizes a Falcon-H1 hybrid backbone (Mamba 2 SSM + attention).

Who it’s for

Developers and researchers looking for a lightweight, multilingual TTS solution that supports zero-shot voice cloning and can be deployed on various hardware, including CPUs via ONNX Runtime or high-performance GPU servers using SGLang Omni serving.

Highlights

  • Zero-Shot Voice Cloning: Mimics voices from a reference audio clip and its transcript.
  • Multilingual Support: Optimized for 11 languages, including English, Chinese, Japanese, Korean, Spanish, French, German, Italian, Dutch, and Polish.
  • Efficient Deployment: Supports CPU deployment via ONNX INT4 and high-throughput serving with SGLang Omni (including paged attention and dynamic batching).
  • Compact Size: Outperforms larger SOTA models on several benchmarks (like Seed-TTS) despite having significantly fewer parameters (0.6B vs 1.5B-8.5B).
  • SFT Pipeline: Includes an independent supervised fine-tuning pipeline for further adaptation.

Related

  • Project
    Saganaki22/ComfyUI-OmniVoice-TTS

    ComfyUI nodes for OmniVoice, providing zero-shot multilingual text-to-speech with voice cloning, voice design, and multi-speaker dialogue support for over 600 languages.

  • Project
    index-tts/index-tts

    A zero-shot text-to-speech system that clones voices from a single audio clip with fine-grained control over emotion, speed, and pronunciation across five languages.

  • Project
    tronghieuit/v-tts

    A lightweight Vietnamese text-to-speech system featuring multi-speaker synthesis and zero-shot voice cloning that runs efficiently on CPU.

  • Project
    OpenMOSS/MOSS-TTS-Nano

    A multilingual tiny speech generation model with 0.1B parameters designed for real-time, CPU-based inference and high-fidelity audio output.

  • Project
    netease-youdao/Confucius4-TTS

    Confucius4-TTS is an LLM-based text-to-speech engine that enables zero-shot, cross-lingual voice cloning across 14 languages without requiring reference transcripts.