resemble-ai/chatterbox

SoTA open-source TTS

What it solves

Chatterbox provides a family of open-source text-to-speech (TTS) models designed to generate natural, conversational audio from text. It addresses the need for high-quality voice cloning and multilingual speech generation that can be deployed across various hardware constraints, from high-performance GPUs to resource-constrained CPUs.

How it works

The project offers several model variants tailored to different use cases:

  • Chatterbox-Turbo: A 350M parameter model optimized for low-latency voice agents, featuring a distilled decoder that reduces generation steps from 10 to one.
  • Chatterbox-Nano: A 110M parameter version of the Turbo architecture designed for on-device and CPU inference, capable of running 3x faster than real-time on 8 CPU cores.
  • Chatterbox-Multilingual V3: A 500M parameter general-purpose model supporting over 23 languages with improved speaker similarity and reduced hallucinations.
  • Single Language Pack: Specialized finetunes for priority languages (e.g., Chinese, Hindi, Spanish) to provide tighter quality control and regional dialect accuracy.

All models support zero-shot voice cloning via a reference audio clip and include built-in imperceptible neural watermarking via the Perth system for responsible AI use.

Who it’s for

  • Developers of voice agents: Those needing low-latency, high-fidelity speech for interactive applications.
  • Localization experts: Users requiring stable cross-language voice cloning across a wide array of languages.
  • Edge device developers: Those deploying TTS on CPUs or devices with tight memory and VRAM budgets.
  • Content creators: Individuals using paralinguistic tags (like [laugh] or [cough]) to add realism to narration.

Highlights

  • Diverse Model Zoo: Options ranging from 110M to 500M parameters to balance speed and quality.
  • Native Paralinguistic Support: Ability to insert non-verbal cues like laughter and chuckles into speech.
  • Broad Multilingualism: Support for 23+ languages with dedicated single-language finetunes for higher precision.
  • CPU-Optimized: Nano model allows for efficient on-device inference.
  • Responsible AI: Integrated neural watermarking to detect generated audio.

Related

  • Project
    devnen/Chatterbox-TTS-Server

    A self-hosted server with a Web UI and OpenAI-compatible API for Resemble AI's Chatterbox TTS family, featuring voice cloning, multilingual support, and audiobook generation.

  • Project
    OpenMOSS/MOSS-TTS-Nano

    A multilingual tiny speech generation model with 0.1B parameters designed for real-time, CPU-based inference and high-fidelity audio output.

  • Project
    OpenMOSS/MOSS-TTS

    A family of open-source speech and sound generation models providing high-fidelity TTS, expressive dialogue, text-to-voice design, real-time voice agents, and sound effect generation.

  • Project
    Audio8-AI/Audio8_TTS

    A compact 0.6B-parameter multilingual text-to-speech model featuring zero-shot voice cloning and efficient deployment options for CPU and GPU.

  • Project
    neuphonic/neutts

    A collection of open-source, on-device text-to-speech models that enable natural-sounding speech and instant voice cloning locally on mobile and embedded devices.