altunenes/parakeet-rs

What it solves

parakeet-rs is a Rust library that provides fast, high-performance speech recognition (ASR) and speaker diarization using NVIDIA's Parakeet models and Cohere Transcribe via the ONNX Runtime. It allows developers to integrate advanced speech-to-text capabilities—including streaming, multilingual support, and multi-speaker identification—directly into Rust applications without needing to manage complex AI model deployments.

How it works

The library acts as a wrapper around ONNX Runtime, enabling the execution of various pre-trained models. It supports multiple model architectures:

• CTC/TDT: For offline transcription with token-level timestamps.
• EOU (End-of-Utterance): For real-time streaming ASR with detection of when a person stops speaking.
• Nemotron: Cache-aware streaming ASR with punctuation and casing, available in English-only and multilingual variants.
• Multitalker: For streaming multi-speaker ASR where transcription is attributed to specific speakers.
• Sortformer: For streaming speaker diarization (identifying who spoke when for up to 4 speakers).
• Cohere Transcribe: For offline multilingual transcription of long-form audio.

Who it’s for

Rust developers who need to implement speech recognition or speaker diarization in their software, particularly those seeking low-latency streaming ASR or high-performance local execution on CPU, CUDA, TensorRT, or WebGPU.

Highlights

• Diverse Model Support: Integrates NVIDIA Parakeet and Cohere models for various use cases (streaming vs. offline).
• Hardware Acceleration: Supports multiple execution providers including CUDA, TensorRT, WebGPU, DirectML, and CPU.
• Casing and Punctuation: Nemotron models provide polished output with proper casing and punctuation.
• Speaker Attribution: Combines ASR with Sortformer for speaker diarization and multi-speaker transcription.
• Token-level Timestamps: Provides precise timing for each word/token in CTC and TDT models.