yxlllc/DDSP-SVC

What it solves

DDSP-SVC provides a way to perform singing voice conversion (AI voice changing) that is significantly more efficient than traditional high-quality methods. It addresses the high hardware requirements and long training times associated with projects like SO-VITS-SVC, making high-quality voice synthesis accessible on standard personal computers.

How it works

The project uses Differentiable Digital Signal Processing (DDSP) and enhances the output quality using advanced models. Current versions utilize a rectified-flow model to improve synthesis quality. The process involves:

1. Preprocessing: Extracting features from audio using encoders like ContentVec or HubertSoft and pitch extraction via RMVPE.
2. Training: Training a model on a specific voice dataset (single or multi-speaker).
3. Synthesis: Converting an input voice to the target voice using either non-real-time processing or a real-time GUI that employs sliding windows, cross-fading, and SOLA-based splicing to maintain low latency.

Who it’s for

It is designed for users who want to create AI voice changers or singing voice conversions but lack professional-grade GPU hardware or the time required for extensive training cycles.

Highlights

• Low Resource Usage: Much lower hardware requirements for training and synthesis compared to SO-VITS-SVC.
• Fast Training: Training speed is orders of magnitude faster, comparable to RVC.
• High Quality: Achieves synthesis quality competitive with SO-VITS-SVC and RVC through the use of rectified-flow models and pre-trained vocoders.
• Flexible Synthesis: Supports both high-fidelity non-real-time conversion and low-latency real-time voice changing.
• Multi-Speaker Support: Ability to train a single model that can switch between or mix different speaker timbres.