pyannote/pyannote-audio

Neural building blocks for speaker diarization: speech activity detection, speaker change detection, overlapped speech detection, speaker embedding

What it solves

It provides a toolkit for speaker diarization, which is the process of partitioning an audio recording into segments and identifying who spoke when (the "who spoke when" problem).

How it works

Built on PyTorch and PyTorch Lightning, the toolkit offers pretrained models and pipelines available via Hugging Face. It supports both local execution (using open-source pipelines like community-1) and cloud-based execution via the pyannoteAI premium service (precision-2). Users can also fine-tune these models on their own data to improve performance for specific use cases.

Who it’s for

Developers and researchers working with audio data who need to accurately identify and separate different speakers in a recording.

Highlights

  • State-of-the-art performance: High accuracy across various benchmarks (e.g., AISHELL-4, AMI, VoxConverse).
  • Flexible deployment: Supports local execution on GPUs or API-based premium services.
  • Pretrained assets: Easy access to models and pipelines through the Hugging Face model hub.
  • Extensible: Supports multi-GPU training and fine-tuning for custom datasets.

Related

  • Project
    BUTSpeechFIT/DiariZen

    A speaker diarization toolkit that uses self-supervised learning and structured pruning to accurately identify who spoke when in audio recordings.

  • Project
    MahmoudAshraf97/whisper-diarization

    A speaker diarization pipeline that combines OpenAI Whisper for transcription with NVIDIA NeMo for speaker identification to determine who spoke which words in an audio file.

  • Project
    speechbrain/speechbrain

    An open-source PyTorch toolkit for Conversational AI that provides recipes and tools for speech and text processing, including speech recognition and synthesis.

  • Project
    kadirnar/whisper-plus

    A comprehensive audio and video processing toolkit that extends OpenAI's Whisper for transcription, speaker diarization, summarization, and RAG-based video chatting.

  • Project
    Uberi/speech_recognition

    A Python library that provides a unified interface for speech recognition, supporting multiple online and offline engines like OpenAI Whisper, Google Speech, and Vosk.