KoljaB/RealtimeSTT

A robust, efficient, low-latency speech-to-text library with advanced voice activity detection, wake word activation and instant transcription.

What it solves

RealtimeSTT simplifies the process of converting spoken audio into text in real-time. It removes the complexity of manually managing voice activity detection (VAD), wake word activation, and audio stream processing, allowing developers to integrate speech-to-text capabilities into assistants or dictation tools with minimal code.

How it works

The library provides an AudioToTextRecorder that can capture audio directly from a microphone or receive PCM audio chunks from external sources. It uses a modular engine system—defaulting to faster_whisper but supporting others like sherpa-onnx and kroko-onnx—to perform transcription. It employs VAD (via WebRTC or Silero) to detect when a user starts and stops speaking and can integrate wake words (via Porcupine or OpenWakeWord) to trigger recording.

Who it’s for

It is designed for developers building voice-controlled AI assistants, dictation software, browser-based streaming servers, and rapid prototypes that require fast, local speech-to-text functionality.

Highlights

  • Multiple Engine Support: Compatible with various backends including faster-whisper, sherpa-onnx, and Kroko-ONNX.
  • Real-time Updates: Supports both immediate text hypotheses and a final authoritative transcript.
  • Flexible Audio Input: Works with live microphones or external audio streams (files, websockets).
  • Production Ready: Includes a packaged FastAPI server with WebSocket support, authentication, and session isolation.
  • Integrated VAD and Wake Words: Built-in support for detecting speech and specific trigger words.

Related

  • Project
    KoljaB/RealtimeTTS

    A Python text-to-speech library designed for low-latency audio generation from strings, generators, and LLM token streams, supporting a wide variety of local and cloud TTS engines.

  • Dispatch
    FastRTC: The Real-Time Communication Library for Python

    Hugging Face has released FastRTC, a Python library designed to simplify the development of real-time audio and video AI applications by handling the WebRTC and WebSocket communication layers.

  • Project
    QuentinFuxa/WhisperLiveKit

    An ultra-low-latency, self-hosted speech-to-text pipeline that enables real-time transcription and translation using advanced simultaneous speech research and multiple model backends.

  • Project
    Uberi/speech_recognition

    A Python library that provides a unified interface for speech recognition, supporting multiple online and offline engines like OpenAI Whisper, Google Speech, and Vosk.

  • Project
    alesaccoia/VoiceStreamAI

    A Python and JavaScript solution for near-real-time audio streaming and transcription using WebSockets, Voice Activity Detection, and Faster Whisper.