kadirnar/whisper-plus

WhisperPlus: Faster, Smarter, and More Capable 🚀

What it solves

WhisperPlus is a comprehensive toolkit for audio and video processing. It simplifies the process of converting speech to text, summarizing the resulting transcripts, and interacting with video content via RAG (Retrieval-Augmented Generation), while also providing tools for speaker diarization and text-to-speech.

How it works

The library provides a set of specialized pipelines that wrap around existing AI models. It integrates with Hugging Face models (like Whisper and BART), Apple MLX for optimized performance on Mac, and other frameworks like Pyannote for speaker identification. It also includes utility functions to download audio/video from YouTube and a RAG system to allow users to "chat" with video transcripts using LLMs.

Who it’s for

Developers and researchers who need a streamlined way to implement speech-to-text, audio summarization, and video-based Q&A systems without building the complex pipelines from scratch.

Highlights

  • Multi-platform Optimization: Supports Apple MLX and Lightning Whisper for efficient Mac execution.
  • Advanced ASR: Includes support for quantization (Hqq, BitsAndBytes) and Flash Attention 2 for faster inference.
  • Full Pipeline Support: Covers the entire workflow from YouTube downloading to transcription, speaker diarization, and auto-captioning.
  • Integrated RAG: Built-in capabilities to chat with video content using LanceDB or AutoLLM.
  • Additional Modalities: Includes text-to-speech (TTS) and text summarization pipelines.

Related

  • Project
    m-bain/whisperX

    A fast automatic speech recognition system that enhances OpenAI's Whisper with batched inference, word-level timestamps, and speaker diarization.

  • Project
    jasoncheng7115/jt-live-whisper

    A fully local AI toolkit for real-time speech transcription, translation, and meeting summarization that works across any system audio source without cloud APIs.

  • Project
    ahmetoner/whisper-asr-webservice

    A general-purpose speech recognition toolkit that wraps Whisper models into a REST API for easy deployment and integration.

  • Project
    wendy7756/AI-Video-Transcriber

    An AI-powered tool to transcribe, summarize, and archive videos and podcasts from 30+ platforms or local files using Faster-Whisper and LLMs.

  • Project
    dimastatz/whisper-flow

    A real-time transcription service that enables streaming speech-to-text using OpenAI Whisper, providing low-latency results instead of batch processing.