RVC-Boss/GPT-SoVITS

1 min voice data can also be used to train a good TTS model! (few shot voice cloning)

What it solves

GPT-SoVITS is designed to create highly realistic text-to-speech (TTS) and voice conversion systems with minimal data. It solves the problem of needing massive datasets to clone a specific person's voice, allowing users to generate speech that sounds like a target speaker using only a few seconds of audio.

How it works

The project uses a combination of GPT and SoVITS architectures to handle voice cloning. It offers two primary modes of operation:

  • Zero-shot TTS: Generates speech using a 5-second vocal sample as a reference without any further training.
  • Few-shot TTS: Fine-tunes the model using approximately 1 minute of training data to significantly improve voice similarity and realism.

It supports cross-lingual inference, meaning it can generate speech in English, Japanese, Korean, Cantonese, and Chinese, even if the training data was in a different language.

Who it’s for

This tool is for creators, developers, and AI enthusiasts who want to clone voices for content creation or application development without requiring professional studio recording sessions or hours of training data.

Highlights

  • Minimal Data Requirements: Clone voices with as little as 5 seconds (zero-shot) or 1 minute (few-shot) of audio.
  • Cross-lingual Support: Supports multiple languages including English, Japanese, Korean, Cantonese, and Chinese.
  • Hỗ trợ WebUI: Includes a comprehensive WebUI for managing the entire pipeline, including audio slicing, denoising, ASR (Automatic Speech Recognition) for transcription, and model training.
  • Hỗ trợ Docker: Provides Docker images for streamlined installation and deployment across different hardware environments.
  • High-speed Inference: Optimized for fast generation, with specific versions (like v2 ProPlus) showing very low Real-Time Factor (RTF) on modern GPUs.

Related

  • Project
    tronghieuit/v-tts

    A lightweight Vietnamese text-to-speech system featuring multi-speaker synthesis and zero-shot voice cloning that runs efficiently on CPU.

  • Project
    Edge0-AI/Audio8_TTS

    A compact multilingual text-to-speech model with zero-shot voice cloning, available in 0.6B and 0.1B parameter versions for high-efficiency speech synthesis.

  • Project
    netease-youdao/Confucius4-TTS

    Confucius4-TTS is an LLM-based text-to-speech engine that enables zero-shot, cross-lingual voice cloning across 14 languages without requiring reference transcripts.

  • Project
    KittenML/KittenTTS

    An open-source text-to-speech library featuring a 1.7B-parameter speech language model with in-context voice cloning and CPU-optimized inference.

  • Project
    neuphonic/neutts

    A collection of open-source, on-device text-to-speech models that enable natural-sounding speech and instant voice cloning locally on mobile and embedded devices.