dimastatz/whisper-flow

Whisper-Flow is a framework designed to enable real-time transcription of audio content using OpenAI’s Whisper model. Rather than processing entire files after upload (“batch mode”), Whisper-Flow accepts a continuous stream of audio chunks and produces incremental transcripts immediately.

What it solves

Whisper Flow transforms OpenAI's Whisper model from a batch-processing tool into a real-time transcription service. It solves the problem of waiting for an entire audio file to be uploaded and processed before receiving a transcript, allowing for immediate, streaming speech-to-text functionality in applications.

How it works

The project uses a "tumbling window" technique to process streaming audio data delivered in sequential packets or "chunks." It splits the audio stream into segments based on natural speech patterns, such as pauses or speaker changes. As the audio is processed, the service returns a series of partial transcription events that are updated until a final segment is completed.

Who it’s for

Developers who want to integrate real-time, low-latency speech-to-text capabilities into their software using the Whisper model, as well as those needing a Python library for managing streaming audio sessions.

Highlights

  • Real-time streaming: Converts audio chunks into text immediately rather than processing files in batch.
  • Low latency: Benchmarked at sub-500ms latency on M1 Mac hardware.
  • Flexible deployment: Can be run as a standalone FastAPI server via WebSockets or integrated as a Python library.
  • PCM Audio Support: Specifically designed for 16 kHz mono 16-bit signed integer audio data.

Related

  • Project
    innovatorved/whisper.api

    A self-hosted, high-performance speech-to-text API powered by whisper.cpp that is Deepgram-compatible for easy integration.

  • Project
    alesaccoia/VoiceStreamAI

    A Python and JavaScript solution for near-real-time audio streaming and transcription using WebSockets, Voice Activity Detection, and Faster Whisper.

  • Project
    m-bain/whisperX

    A fast automatic speech recognition system that enhances OpenAI's Whisper with batched inference, word-level timestamps, and speaker diarization.

  • Project
    QuentinFuxa/WhisperLiveKit

    An ultra-low-latency, self-hosted speech-to-text pipeline that enables real-time transcription and translation using advanced simultaneous speech research and multiple model backends.

  • Project
    jasoncheng7115/jt-live-whisper

    A fully local AI toolkit for real-time speech transcription, translation, and meeting summarization that works across any system audio source without cloud APIs.