collabora/WhisperLive

WhisperLive – Near‑real‑time transcription with OpenAI’s Whisper

What it is – WhisperLive is an open‑source application that turns spoken audio into text almost as it is spoken. It runs a server that hosts a Whisper model (via the faster‑whisper, TensorRT, or OpenVINO back‑ends) and one‑or‑more clients that stream microphone, file, RTSP, HLS or raw PCM audio and receive transcriptions over WebSocket or a REST API.

Key capabilities

• Live and batch modes – stream microphone input or transcribe whole audio files.
• Multiple inference back‑ends –
  • faster‑whisper (CPU/GPU via CTranslate2)
  • NVIDIA TensorRT (high‑throughput GPU inference, optional Docker setup)
  • Intel OpenVINO (CPU, iGPU, dGPU)
  • AMD ROCm support for CTranslate2.
• Advanced output options
  • Word‑level timestamps and confidence scores.
  • Custom vocabulary / hot‑word boosting.
  • Speaker diarization (optional pyannote.audio).
  • Optional translation of the transcript to another language.
• Batch inference – combine several client streams into a single GPU call for higher throughput.
• Client APIs – a high‑level TranscriptionClient for simple file/mic use and a StreamingTranscriptionClient for manual chunked audio feeding with callbacks for partial and final results.
• Cross‑platform – Linux, macOS, Windows (via Docker for GPU back‑ends). Also ships a Chrome/Firefox extension and a native iOS client.
• Docker images – ready‑to‑run containers for GPU (TensorRT, OpenVINO, ROCm) and CPU deployments.

Typical workflow

1. Install system deps – portaudio (for microphone) and Python 3.12.
2. Create a virtual environment and pip install whisper-live.
3. Start the server (example with the faster‑whisper back‑end):

   python3 run_server.py --port 9090 --backend faster_whisper \
       --max_clients 4 --max_connection_time 600
   

4. Run a client – transcribe a file, microphone, RTSP or HLS stream:

   python3 run_client.py --files mytalk.wav
   # or live mic
   python3 run_client.py
   

   Or use the Python API:

   from whisper_live.client import TranscriptionClient
   client = TranscriptionClient("localhost", 9090, model="small", translate=True, target_language="hi")
   client("tests/jfk.wav")
   

5. Optional features – add word_timestamps=True, hotwords="WhisperLive,TensorRT", or enable_diarization=True when constructing the client.

Installation snippet

bash scripts/setup.sh               # installs portaudio dev libs
python3.12 -m venv whisper_env && source whisper_env/bin/activate
pip install whisper-live

Running with hardware acceleration

• TensorRT – build a TensorRT engine (see TensorRT_whisper.md) and start the server with --backend tensorrt and --trt /path/to/engine.
• OpenVINO – install OpenVINO runtime, then python3 run_server.py -p 9090 -b openvino (Docker image ghcr.io/collabora/whisperlive-openvino handles drivers automatically).
• ROCm – follow ROCm_whisper.md for AMD GPU support.

Docker quick‑start (GPU)

docker run -it --gpus all -p 9090:9090 ghcr.io/collabora/whisperlive-gpu:latest

(Replace the image tag for TensorRT, OpenVINO, or ROCm as needed.)

Why it matters – WhisperLive makes the powerful Whisper speech‑to‑text model usable in production‑style low‑latency scenarios (live captioning, meeting transcription, voice‑controlled apps) while offering flexibility in hardware back‑ends, language translation, diarization and custom vocabularies.

Further reading – Blog posts linked in the repo detail production use‑cases and the WhisperFusion chatbot integration.