altunenes/parakeet-rs

parakeet‑rs – Fast speech‑to‑text in Rust

What it is – parakeet‑rs is a Rust crate that lets you run NVIDIA’s Parakeet speech‑recognition models (CTC, TDT, streaming, multilingual, diarization, etc.) through the ONNX Runtime. It provides a thin, ergonomic API for loading the ONNX files, feeding 16 kHz mono audio, and getting back transcribed text together with optional timestamps and speaker labels.

Why it matters – The original Parakeet models are large, high‑quality ASR models that run on GPU. parakeet‑rs makes them usable from native Rust programs without pulling in Python or heavyweight deep‑learning frameworks, and it supports a range of execution providers (CUDA, TensorRT, WebGPU, DirectML, MIGraphX, CPU). This enables low‑latency, on‑device transcription for desktop, server, or embedded Rust applications.

Key capabilities

• CTC (English‑only) – fast offline transcription with word‑level timestamps.
• TDT (multilingual) – 25‑language auto‑detect model, sentence‑level timestamps.
• EOU (end‑of‑utterance) – real‑time streaming ASR that stops when speech ends.
• Nemotron streaming – cache‑aware streaming with punctuation; English‑only (0.6 B) and multilingual (3.5 B) variants.
• Cohere Transcribe – optional offline multilingual model (14 languages) with optional punctuation and inverse‑text‑normalisation.
• Multitalker – streaming multi‑speaker transcription; each chunk returns a list of (speaker_id, text) pairs.
• Sortformer diarization – separate speaker‑diarization models (up to 4 speakers) that can be run offline or chunk‑wise for real‑time use.
• Token‑level timestamps – for CTC/TDT models you can retrieve start/end times per token.
• GPU/CPU flexibility – select the ONNX Runtime execution provider via Cargo features (cuda, tensorrt, webgpu, directml, migraphx, etc.) or fall back to CPU automatically.

Typical usage (simplified)

use parakeet_rs::{Parakeet, Transcriber, TimestampMode};

// Load a pretrained CTC model from a local directory
let mut model = Parakeet::from_pretrained("./ctc", None)?;

// `audio` = Vec<f32> of 16 kHz mono samples
let result = model.transcribe_samples(audio, 16000, 1, Some(TimestampMode::Words))?;
println!("Transcribed: {}", result.text);

// Optional per‑token timestamps
for token in result.tokens {
    println!("[{:.3}s – {:.3}s] {}", token.start, token.end, token.text);
}

The crate ships with a set of example programs (examples/*.rs) that demonstrate each model type, streaming loops, and diarization pipelines.

Getting the models The library does not ship the ONNX weights; you must download them from the linked Hugging Face repositories (e.g., parakeet-ctc-0.6b-ONNX, parakeet-tdt-0.6b-v3-onnx, nemotron‑3.5‑asr‑streaming‑0.6b). The README provides direct URLs and a small Python helper (scripts/download_orukeet.py) that verifies hashes and caches the files locally.

Installation

# Cargo.toml
parakeet-rs = { version = "0.3", features = ["cuda"] }   # or "webgpu", "tensorrt", etc.

Then configure the execution provider if you need something other than the default CPU:

use parakeet_rs::{Parakeet, ExecutionConfig, ExecutionProvider};
let cfg = ExecutionConfig::new().with_execution_provider(ExecutionProvider::Cuda);
let mut model = Parakeet::from_pretrained("./model", Some(cfg))?;

Advanced users can tweak the underlying ONNX Runtime SessionBuilder via ExecutionConfig::with_custom_configure.

License – The code is dual‑licensed under MIT or Apache‑2.0. The ONNX model files are not included; they remain under NVIDIA’s or the respective model owners’ licenses (e.g., CC BY‑SA 4.0 for the Orukeet fine‑tuned model).

---

All details above are taken directly from the repository’s README; no additional features have been inferred.