SYSTRAN/faster-whisper

What it is

faster‑whisper is a Python library that runs OpenAI’s Whisper speech‑to‑text model — but does it with the high‑performance inference engine CTranslate2. By re‑implementing the model on top of CTranslate2 it can transcribe up to four times faster than the original openai/whisper code while using roughly the same amount of memory. The package works on CPU, GPU and even supports 8‑bit quantisation to squeeze out extra speed.

Why it matters

Whisper is a popular, high‑quality model for turning audio into text, but the reference implementation is relatively slow and memory‑hungry, especially for the larger model sizes. Faster‑whisper makes the same accuracy reachable in real‑time or near‑real‑time scenarios (e.g., live captioning, batch processing of long recordings) without needing specialised hardware beyond a modern GPU or a decent multi‑core CPU. The speed‑up also lowers cloud‑compute costs for large‑scale transcription pipelines.

How to get started

1. Install – the library is on PyPI:

   pip install faster-whisper
   

   (Optional: install a specific commit or the master branch via the URLs shown in the README.)
2. Choose a model size – e.g., "large-v3", "distil-large-v3", or any Whisper size supported on the Hugging Face Hub.
3. Create a model specifying the device and precision:

   from faster_whisper import WhisperModel
   model = WhisperModel("large-v3", device="cuda", compute_type="float16")
   

4. Transcribe an audio file:

   segments, info = model.transcribe("audio.mp3", beam_size=5)
   print(info.language, info.language_probability)
   for s in segments:
       print(f"[{s.start:.2f}s -> {s.end:.2f}s] {s.text}")
   

   segments is a generator, so the transcription runs when you iterate over it.
5. Optional extras – enable word‑level timestamps, VAD‑based silence removal, or batch transcription for higher throughput (see the README for the exact arguments).

When to use it

• Live or near‑live captioning – the GPU‑batch mode can finish a 13‑minute video in ~16 s.
• Batch processing of large audio collections – CPU int8 mode gives a good speed‑memory trade‑off.
• Projects that need Whisper‑compatible output but cannot afford the original runtime – e.g., web services, desktop apps, or research pipelines.
• When you want to experiment with distilled Whisper models – the library directly supports distil‑large‑v3 checkpoints.

Limitations & gotchas

• GPU requirements – you need CUDA 12, cuBLAS and cuDNN 9 (or downgrade ctranslate2 for older CUDA versions). Installing these libraries can be non‑trivial on Windows.
• Memory usage – the fastest batch modes increase VRAM (up to ~6 GB for batch‑size 8 on a 3070 Ti). Plan GPU resources accordingly.
• Default beam size – the library defaults to a beam size of 5, which is larger than the default in openai/whisper. Larger beams improve accuracy but increase latency.
• Model conversion – if you want to run a custom or fine‑tuned Whisper checkpoint you must first convert it to the CTranslate2 format using the provided ct2‑transformers‑converter script.
• Silero VAD – VAD filtering is enabled by default for batched transcription; it may drop very short pauses that some applications consider speech.

---

TL;DR: Faster‑whisper is a drop‑in, faster, lower‑memory Python wrapper around Whisper that leverages CTranslate2. Install via pip, pick a model, and call model.transcribe. It shines for real‑time or large‑scale transcription, but you need compatible CUDA libraries for GPU use and may need to convert custom models.