chenpipi0807/ComfyUI-Index-TTS

What it solves

This project provides a set of custom nodes for ComfyUI that enable high-quality text-to-speech (TTS) conversion. It allows users to clone voice characteristics from a short reference audio clip and generate speech with precise control over language, speed, and emotion, solving the difficulty of creating natural, multi-lingual, and emotionally expressive synthetic voices within a node-based workflow.

How it works

The project implements the IndexTTS family of models (versions 1.5, 2, and 2.5) as ComfyUI nodes. It uses a reference audio file to extract speaker embeddings and combines this with input text to synthesize speech.

Key technical mechanisms include:

• Multi-model Support: Integration of IndexTTS-2.5, 2, and 1.5.
• Emotion Control: Three methods for directing emotion: using a reference emotion audio clip, adjusting an 8-dimensional emotion vector (e.g., happiness, anger, sadness), or providing a text description of the emotion (powered by a Qwen emotion model).
• Pronunciation Control: In version 2.5, users can insert specific phonetic markers (Pinyin for Chinese, CMU phonemes for English, and Kana for Japanese) directly into the text to correct pronunciation.
• Novel Parsing: A specialized "Novel Text Structure" node that automatically identifies characters and narrators in a story to assign different voices to different roles.
• Audio Cleaning: An integrated Audio Cleaner node to remove reverb and noise from the output.

Who it’s for

• Content Creators: Those making audiobooks, podcasts, or videos who need consistent, high-quality voice cloning.
• ComfyUI Users: AI artists and developers who want to integrate professional TTS into their generative pipelines.
• Storytellers: Users wanting to generate multi-character dialogue for novels or scripts.

Highlights

• Cross-lingual Cloning: Ability to use a Chinese reference audio to generate speech in English, Japanese, Spanish, or Arabic (IndexTTS-2.5).
• Precise Speed Control: Native duration_factor for adjusting speech rate without sacrificing quality.
• Multi-Role Support: Dedicated nodes for parsing novel text and assigning up to five distinct character voices plus a narrator.
• Flexible Emotion Tuning: Support for both vector-based sliders and natural language emotion descriptions.
• ** uma Memory Management**: A Cache Control node to manage VRAM usage by toggling whether models remain resident in memory.