Reachy Mini Local Speech Backend Integration

Hugging Face has introduced a method to run the Reachy Mini conversation stack fully locally, removing the requirement to send audio to a remote server. This is achieved using the speech-to-speech library, which implements a cascaded pipeline of Voice Activity Detection (VAD), Speech-to-Text (STT), a Large Language Model (LLM), and Text-to-Speech (TTS) that exposes a Realtime API-compatible /v1/realtime WebSocket.

Local Pipeline Components and Recommended Defaults

The speech-to-speech library utilizes a cascaded approach, allowing users to swap individual components based on their needs for latency, quality, or language support. Hugging Face provides the following opinionated defaults for the voice pipeline:

Stage	Choice	Rationale
VAD	Silero VAD v5	Tiny, accurate, and runs efficiently on CPU.
STT	Parakeet-TDT 0.6B v3	Fast, streaming-friendly, and high quality for English.
TTS	Qwen3-TTS	Expressive, low-latency, multilingual, and supports custom voices.

LLM Deployment Options

The LLM is the primary driver of system latency. The speech-to-speech engine supports multiple deployment strategies to decouple the "brain" from the voice loop via the Responses API protocol.

Local Inference Engines

Users can run models locally using several backends:

• llama.cpp: Recommended for general local use. For example, running gemma-4-E4B-it-GGUF with flash attention (-fa on) and full sliding-window attention cache (--swa-full) to optimize prompt processing.
• MLX (Apple Silicon): The lowest-friction option for Mac users, with Qwen3-4B-Instruct-2507 recommended for a balance of speed and capability.
• Transformers: Suitable for CUDA, Linux, or users who want to swap models without weight conversion.
• vLLM (v0.21.0+): Supports the Responses API protocol and tool-call streaming. Key configurations include --enable-auto-tool-choice, a specific --tool-call-parser, and disabling the reasoning channel (--default-chat-template-kwargs '{"enable_thinking":false}') to prevent audible silence during conversation.

Hosted and Managed Endpoints

For those preferring managed infrastructure, the pipeline supports:

• Hugging Face Inference Endpoints: Deploying a chat model on a managed GPU.
• Hugging Face Inference Providers: Routing requests to third-party backends (e.g., Together, Fireworks, Replicate) via a single URL.
• OpenAI and Compatible Providers: Pointing the pipeline at OpenAI or any provider implementing the same protocol (e.g., OpenRouter).

Technical Implementation and Connectivity

To implement the local stack, users install the speech-to-speech library via uv pip install speech-to-speech. The system can be run in two primary modes: --mode local for terminal-based testing and --mode realtime to serve the robot.

When the voice engine is hosted on a laptop and the conversation app runs on a Reachy Mini Wireless, the engine must bind to a LAN address (e.g., 192.168.x.x) rather than 127.0.0.1. The robot's UI is then configured to point to the laptop's IP address via the "edit connection" setting in the HF backend.

Advantages of Local Deployment

Running a local speech-to-speech server provides three primary benefits:

1. Privacy: Audio data remains within the local network and on controlled hardware.
2. Cost: Eliminates per-minute or per-token API fees.
3. Control: Allows the full swapping of VAD, STT, LLM, and TTS components as new models are released on the Hugging Face Hub.