kyutai-labs/unmute

What it solves

Unmute enables standard text-based Large Language Models (LLMs) to engage in real-time voice conversations. It bridges the gap between text LLMs and spoken interaction by integrating high-performance speech-to-text (STT) and text-to-speech (TTS) models, reducing the latency typically associated with voice interfaces.

How it works

Unmute acts as an orchestration layer that wraps a text LLM with specialized audio models:

1. Speech-to-Text: Transcribes user audio input in real time via a websocket connection.
2. LLM Processing: Once the STT detects the user has finished speaking, the transcribed text is sent to an LLM (such as Gemma 3 or models via OpenRouter/vLLM) to generate a text response.
3. Text-to-Speech: The LLM's text output is streamed into a TTS engine, which converts it back into audio and forwards it to the user.

Who it’s for

• Developers looking to add low-latency voice capabilities to their existing text LLMs.
• Researchers experimenting with real-time audio-text-audio pipelines.
• Self-hosters who want a private, voice-enabled AI assistant using open-source models.

Highlights

• Low Latency: Optimized STT and TTS components to ensure fluid conversation.
• LLM Agnostic: Works with any OpenAI-compatible LLM server (vLLM, Ollama, OpenRouter).
• Flexible Deployment: Supports Docker Compose for easy setup, Dockerless for manual control, and Docker Swarm for scaling.
• Customizable Voices: Allows users to change characters and voices via a configuration file.