narcotic-sh/senko

What it solves

Senko is a high-performance speaker diarization pipeline designed to identify "who spoke when" in an audio recording. It focuses on extreme speed and accuracy, enabling the processing of one hour of audio in just a few seconds on modern hardware (e.g., 5 seconds on an RTX 4090).

How it works

Senko is an optimized version of the 3D-Speaker pipeline, utilizing a four-stage process:

1. Voice Activity Detection (VAD): Uses either Pyannote segmentation-3.0 or Silero VAD to find speech segments.
2. Feature Extraction: Extracts Fbank features, utilizing GPU acceleration via kaldifeat on NVIDIA systems.
3. Embedding Generation: Generates speaker embeddings using the CAM++ model with batched inference.
4. Clustering: Groups similar embeddings using spectral clustering or UMAP+HDBSCAN, with GPU acceleration via RAPIDS on supported NVIDIA cards.

On Mac, it leverages CoreML for VAD and embeddings to optimize performance on Apple Silicon.

Who it’s for

Developers and researchers who need a fast, accurate, and language-agnostic speaker diarization tool that can be integrated into transcription services or media players.

Highlights

• Extreme Speed: Processes 1 hour of audio in under 8 seconds on high-end GPUs and Apple M3 chips.
• GPU Acceleration: Full GPU pipeline on Linux/WSL for NVIDIA users (CUDA 7.0+).
• ** uma-agnostic:** Works across various languages by focusing on acoustic patterns, though optimized for English and Mandarin.
• Cross-Platform Support: Native support for Linux, macOS, and Windows (via WSL).
• Integration Ready: Provides JSON and RTTM output formats for easy integration into other applications.