Softcatala/whisper-ctranslate2

What it solves

Whisper-ctranslate2 is a high-performance command-line client for OpenAI's Whisper, designed to reduce the time and memory required for audio transcription and translation. It addresses the slow inference speeds of the original OpenAI implementation by leveraging CTranslate2 and Faster-whisper.

How it works

The project uses the CTranslate2 engine and the Faster-whisper implementation of Whisper models. It supports various hardware optimizations (Intel MKL, oneDNN, OpenBLAS, Ruy, and Apple Accelerate) and quantization (e.g., int8) to speed up execution on both CPU and GPU. It maintains compatibility with the original OpenAI Whisper CLI for easy migration.

Who it’s for

Users who need fast, efficient audio-to-text transcription and translation, particularly those running on limited hardware or processing large volumes of audio files.

Highlights

• High Performance: Up to 4x faster than the original OpenAI Whisper, with further speed increases (up to 16x) via batched inference.
• Speaker Diarization: Integrates with pyannote.audio to identify different speakers in an audio recording.
• VAD Filter: Uses Silero VAD to filter out non-speech segments to improve transcription quality.
• Live Transcription: Supports real-time transcription directly from a microphone.
• Flexible Deployment: Provided as a Python package or a Docker image with pre-loaded models.
• Custom Models: Ability to load fine-tuned or quantized Whisper models from a local directory.