fishaudio/fish-diffusion

An easy to understand TTS / SVS / SVC framework

What it solves

Fish Diffusion is a training framework designed to simplify the creation of high-quality voice generation models. It addresses the complexity of training Text-to-Speech (TTS), Singing Voice Synthesis (SVS), and Singing Voice Conversion (SVC) models by providing a decoupled, easy-to-understand code structure.

How it works

The project utilizes Diffusion Models to handle various voice generation tasks. It integrates with vocoders like FishAudio NSF-HiFiGAN and supports the extraction of audio features such as pitch, text, and mel features to train models that can generate or convert voices.

Who it’s for

It is intended for developers and researchers working on voice synthesis and conversion, specifically those looking for a framework that supports multi-speaker training and efficient hardware utilization.

Highlights

  • Multi-speaker support: Capable of training models that handle multiple different voices.
  • Efficient training: Supports half-precision training and multi-machine, multi-device setups to reduce memory usage and increase speed.
  • Modular design: Features a decoupled code structure that is simpler to understand than previous iterations like diffsvc.
  • Broad compatibility: Supports the 44.1khz Diff Singer community vocoder and provides tools to convert existing DiffSVC models.

Related

  • Project
    MoonInTheRiver/DiffSinger

    A diffusion-based framework for high-quality singing voice synthesis and text-to-speech generation from lyrics, MIDI, and pitch data.

  • Project
    SWivid/F5-TTS

    F5-TTS is a high-performance text-to-speech system using a Diffusion Transformer and flow matching to generate fluent, faithful speech with fast training and inference.

  • Project
    prophesier/diff-svc

    A singing voice conversion tool that uses diffusion models to transform input singing voices into a target timbre with basic pitch correction support.

  • Project
    fishaudio/fish-speech

    A state-of-the-art multilingual text-to-speech system that uses a Dual-AR architecture to provide realistic voice cloning and fine-grained emotional control via natural language tags.

  • Project
    yxlllc/DDSP-SVC

    An efficient singing voice conversion project that enables high-quality AI voice changing with significantly lower hardware requirements and faster training times than traditional SVC models.