RVC-Project/Retrieval-based-Voice-Conversion-WebUI

Easily train a good VC model with voice data <= 10 mins!

What it solves

This project provides a simple and easy-to-use framework for voice timbre conversion and voice changing. It allows users to transform a source voice into a target voice while maintaining the original speech patterns, effectively acting as a high-quality voice changer.

How it works

The system uses a retrieval-based approach to replace input source features with features from a training set (top1 retrieval) to prevent timbre leakage. It leverages the RMVPE (InterSpeech2023) pitch extraction algorithm to eliminate mute sounds and ensure fast, low-resource processing. The framework also includes tools for vocal separation (via pymss/MSST) and model merging to adjust timbres.

Who it’s for

It is designed for creators and developers who want to train their own voice models with minimal data (as little as 10 minutes of low-noise audio) and perform either offline conversion or real-time voice changing.

Highlights

  • Low Latency: Achieves end-to-end latency of 170ms, which can be reduced to 90ms using ASIO devices.
  • Efficient Training: Capable of fast training even on lower-end GPUs.
  • High Quality: Uses RMVPE for precise pitch extraction and top1 retrieval to ensure voice consistency.
  • Accessibility: Includes a user-friendly WebUI and supports multiple hardware configurations, including NVIDIA, AMD, and Intel (via DirectML/CPU).
  • Integrated Tools: Built-in support for separating vocals from accompaniment.

Related

  • Project
    prophesier/diff-svc

    A singing voice conversion tool that uses diffusion models to transform input singing voices into a target timbre with basic pitch correction support.

  • Project
    yxlllc/DDSP-SVC

    An efficient singing voice conversion project that enables high-quality AI voice changing with significantly lower hardware requirements and faster training times than traditional SVC models.

  • Project
    Picovoice/speech-to-text-benchmark

    A minimalist framework for benchmarking speech-to-text engines across various datasets to compare accuracy, latency, and computational efficiency.

  • Project
    voicepaw/so-vits-svc-fork

    A singing voice conversion tool that enables real-time voice changing and high-quality voice cloning with an improved GUI and faster training.

  • Project
    Stability-AI/stable-codec

    A high-quality, low-bitrate speech codec that compresses audio into discrete tokens, optimized for use in downstream tasks like text-to-speech.