ASLP-lab/MeanVC

A Lightweight and Streaming Zero-Shot Voice Conversion via Mean Flows

What it solves

MeanVC addresses the challenge of performing high-fidelity, zero-shot voice conversion—changing the timbre of a speaker's voice while keeping the spoken words—in real-time. It aims to reduce the computational overhead and latency typically associated with diffusion-based voice conversion, making it lightweight enough for streaming applications.

How it works

The system uses a diffusion transformer combined with a chunk-wise autoregressive denoising strategy to process audio in streams. To achieve fast generation, it employs "mean flows," which allow the model to map the starting point to the endpoint in a single step, bypassing the need for multiple iterative denoising steps. It can convert a source voice to any target voice without needing to be re-trained on that specific speaker (zero-shot).

Who it’s for

This tool is designed for researchers and developers working on speech synthesis, real-time audio processing, and voice cloning applications where low latency and high speaker similarity are required.

Highlights

  • Streaming Inference: Supports real-time conversion via microphone input using chunk-wise processing.
  • Single-Step Generation: Uses mean flows for rapid inference compared to traditional diffusion models.
  • Zero-Shot Capability: Transfers timbre to unseen target speakers without additional training.
  • Lightweight Architecture: Uses significantly fewer parameters than previous methods while maintaining high speech quality.

Related

  • Project
    prophesier/diff-svc

    A singing voice conversion tool that uses diffusion models to transform input singing voices into a target timbre with basic pitch correction support.

  • Project
    yxlllc/DDSP-SVC

    An efficient singing voice conversion project that enables high-quality AI voice changing with significantly lower hardware requirements and faster training times than traditional SVC models.

  • Project
    dimastatz/whisper-flow

    A real-time transcription service that enables streaming speech-to-text using OpenAI Whisper, providing low-latency results instead of batch processing.

  • Project
    RVC-Project/Retrieval-based-Voice-Conversion-WebUI

    A voice timbre conversion framework that uses retrieval-based features to transform voices with high fidelity and low latency.

  • Project
    fishaudio/fish-diffusion

    A diffusion-based training framework for Text-to-Speech, Singing Voice Synthesis, and Singing Voice Conversion that supports multi-speaker training and efficient hardware utilization.