KoljaB/RealtimeTTS

What it solves

RealtimeTTS addresses the high latency typically associated with text-to-speech (TTS) systems, specifically for applications that need to convert strings, generators, or live LLM token streams into audio in real-time. It provides a unified interface to handle the complexities of streaming audio playback and engine management.

How it works

The library acts as a wrapper around a vast matrix of TTS engines, allowing developers to feed text into a stream that can be played locally, streamed to another process, or saved as a WAV file. It supports both synchronous and asynchronous playback and can handle text arriving in chunks (via iterators) to ensure audio begins playing as soon as possible. It also includes sentence splitting logic to optimize the flow of speech.

Who it’s for

It is designed for developers building conversational AI, voice assistants, or any application requiring low-latency, high-quality speech synthesis from dynamic text sources.

Highlights

• Extensive Engine Support: Integrates with local neural models (like QwenEngine and Coqui), cloud APIs (OpenAI, Azure, Elevenlabs), and system voices.
• Low Latency: Optimized for fast "Time to First Token" (TTFT) and minimal audible onset, particularly with the recommended QwenEngine.
• Flexible Output: Supports local speaker playback, WAV file export, and a headless HTTP server mode for production environments.
• Advanced Controls: Includes features for voice cloning, fallback engines for reliability, and detailed callbacks for word-timing and audio chunks.
• LLM Integration: Specifically designed to handle token streams from LLMs to minimize the gap between text generation and speech.