sp-uhh/sgmse
Score-based Generative Models (Diffusion Models) for Speech Enhancement and Dereverberation
What it solves
This project provides a framework for speech enhancement and dereverberation, addressing the problem of removing noise and reverberation from audio signals to restore clean speech. It implements several research papers on using diffusion-based generative models to achieve this restoration.
How it works
The system uses diffusion-based generative models (specifically score-based generative models) that operate in the complex Short-Time Fourier Transform (STFT) domain. It employs a forward process where noise is gradually added to a clean speech spectrogram, and a reverse process that learns to iteratively generate clean speech from a corrupted signal.
Who it’s for
This is primarily aimed at researchers and developers in the audio signal processing and machine learning community who are working on speech restoration, generative AI for audio, and diffusion models.
Highlights
- Multiple Model Architectures: Supports various backbones such as
ncsnpp,dcunet, andncsnpp_v2. - Comprehensive Pretrained Checkpoints: Offers checkpoints for various tasks (enhancement and dereverberation) trained on datasets like VoiceBank-DEMAND, WSJ0-CHiME3, and EARS.
- Fullband Support: Includes models capable of handling 48 kHz audio.
- Evaluation Tools: Includes scripts for generating enhanced audio files and calculating instrumental metrics to measure performance.
- Advanced Generative Approaches: Implements the Schrödinger bridge model and investigates different training objectives for speech restoration.