Saganaki22/ComfyUI-OmniVoice-TTS

What it solves

This project provides ComfyUI nodes for OmniVoice, a zero-shot multilingual text-to-speech (TTS) system. It enables users to generate high-quality speech in over 600 languages, allowing for precise voice cloning from short audio samples or the creation of entirely synthetic voices based on text descriptions.

How it works

The system uses a diffusion-based audio generation process. It integrates several specialized nodes for different TTS tasks:

• Voice Cloning: Uses a 3-15 second reference audio clip to mimic a specific voice.
• Voice Design: Creates new voices by specifying attributes like gender, age, pitch, and accent via text instructions.
• Longform TTS: Handles extended text with smart chunking to maintain stability.
• Multi-Speaker TTS: Generates conversations by using [Speaker_N]: tags to assign lines to different voices.
• Whisper Integration: Uses OpenAI's Whisper ASR to automatically transcribe reference audio for better cloning accuracy.

To optimize performance, it supports SageAttention CUDA kernels for faster inference on compatible GPUs (SM80+), automatic CPU offloading to save VRAM, and multiple precision modes (fp32, bf16, fp16).

Who it’s for

Content creators, developers, and AI artists using ComfyUI who need professional-grade, multilingual voice synthesis, voice cloning, and dialogue generation without needing to train their own models.

Highlights

• Massive Language Support: Supports 600+ languages.
• Zero-Shot Cloning: Clones voices from just 3-15 seconds of audio.
• Expressive Control: Includes non-verbal tags (e.g., [laughter], [sigh]) and specific dialect/style instructions.
• High Efficiency: Offers inference speeds up to 40x faster than real-time (RTF 0.025).
• VRAM Flexible: Supports automatic CPU offload and quantized models (bf16) to run on lower-end hardware.