KoljaB/RealtimeTTS

Converts text to speech in realtime

What it solves

RealtimeTTS addresses the high latency typically associated with text-to-speech (TTS) systems, making it suitable for conversational AI applications. It allows developers to convert strings, generators, and live LLM token streams into audio in real-time, reducing the gap between text generation and audible speech.

How it works

The library acts as a flexible wrapper around a vast matrix of TTS engines. It can process text inputs (including streaming chunks from an LLM) and feed them into a selected engine to produce audio. It supports local playback, streaming audio chunks to other processes, or saving output to WAV files. To minimize latency, it utilizes techniques like sentence splitting and supports high-performance engines like Qwen, which can run on GPU or CPU with optimized native runtimes.

Who it’s for

It is designed for Python developers building voice-enabled AI agents, conversational bots, and any application requiring low-latency, high-quality speech synthesis from dynamic text sources.

Highlights

  • Broad Engine Support: Integrates with local system voices, cloud APIs (OpenAI, Azure, ElevenLabs), and local neural models (Qwen, Coqui, Piper).
  • Low Latency: Optimized for streaming LLM output to ensure fast audible onset.
  • Voice Cloning: Supports x-vector and ICL voice cloning via compatible engines like Qwen.
  • Flexible Output: Supports local speaker playback, WAV file export, and muted synthesis.
  • Resilience: Includes fallback engine capabilities to ensure synthesis continues if a primary engine fails.

Related

  • Project
    KoljaB/RealtimeSTT

    A Python speech-to-text library that provides fast, real-time transcription with integrated voice activity detection and wake word support.

  • Project
    KittenML/KittenTTS

    An open-source text-to-speech library featuring a 1.7B-parameter speech language model with in-context voice cloning and CPU-optimized inference.

  • Project
    dbccccccc/ttsfm

    An OpenAI-compatible text-to-speech API service and Python SDK that converts text to natural speech using the openai.fm backend.

  • Project
    kyutai-labs/unmute

    A system that enables text LLMs to listen and speak by wrapping them in low-latency speech-to-text and text-to-speech models.

  • Project
    huggingface/speech-to-speech

    speech‑to‑speech is an open‑source Python toolkit that runs a full voice‑assistant locally. It chains VAD → speech‑to‑text → LLM → text‑to‑speech, with many interchangeable model back‑ends, and speaks the OpenAI Realtime protocol so existing clients can connect to your own machine.