transcribe.cpp v0.1.0: A Cross-Platform GGML-Based ASR Inference Engine

High-Performance Local ASR Inference

transcribe.cpp v0.1.0 is a ggml-based transcription library designed to provide fast, accurate, and cross-platform Automatic Speech Recognition (ASR) inference. It supports over 60 models across 16 ASR families and is engineered to be a lightweight, embeddable alternative to heavy frameworks like PyTorch or complex distribution stacks involving multiple engines.

Key capabilities of the library include:

  • Broad Hardware Acceleration: Support for Vulkan, Metal, CUDA, and TinyBLAS.
  • Versatile Transcription Modes: Support for both streaming and batch transcription.
  • Multi-Language Bindings: First-party maintained bindings for Python, JavaScript/TypeScript, Rust, and ObjC/Swift.
  • Cross-Platform Compatibility: Native operation on macOS, Windows, and Linux.

Solving the ASR Distribution Problem

Developing cross-platform applications with local ASR has historically been difficult due to a fragmented inference stack. Developers often had to choose between whisper.cpp and ONNX, or maintain separate engines (such as MLX for Apple devices) to achieve optimal performance.

transcribe.cpp addresses these pain points by providing a single, trusted engine that can run various SOTA models on the GPU across different operating systems. It is designed as a more or less drop-in replacement for whisper.cpp, maintaining compatibility with popular .bin files to ease migration for existing projects.

Rigorous Numerical Validation and WER Testing

To eliminate the uncertainty often associated with converted models (such as .onnx files found on Hugging Face), transcribe.cpp implements a strict validation pipeline:

  1. Numerical Validation: Every supported model is numerically validated against its reference implementation to ensure the engine's output matches the original.
  2. Word Error Rate (WER) Sweeps: The library conducts full WER sweeps across thousands of utterances to verify that the final transcription remains consistent with the reference implementation.

These validation results are published both in the transcribe.cpp repository and alongside the models on the Hugging Face handy-computer organization.

Integration and Ecosystem

transcribe.cpp was developed to power Handy, a cross-platform dictation application. Because it was built for a real-world production app, the library prioritizes embeddability and distribution.

Community Insights and Use Cases

Users and developers in the community have highlighted several practical applications and current limitations of the local ASR landscape:

  • Low-Power Hardware: Users report successful near-real-time offline STT on devices as old as 10-year-old phones and RK3566 CPUs.
  • Technical Vocabulary: A common challenge remains the "injection" of technical vocabulary to prevent the misinterpretation of domain-specific terms (e.g., "OpenBao" being transcribed as "open bowel").
  • Feature Requests: Community members have expressed interest in integrated Voice Activity Detection (VAD), speaker diarization (speaker identification), and support for the International Phonetic Alphabet (IPA) for linguistic research on minority languages.

"I've been using this one for a week for local transcription, working pretty well so far"

"The numerical validation and WER testing are what stand out to me here. A lot of local ASR projects claim broad model support, but it is often difficult to know whether the converted models still match their reference implementations."

Technical Acknowledgments

The development of transcribe.cpp was supported by several organizations:

  • Mozilla AI: Provided support through the BiR program.
  • ggml: The foundational library that enables the distribution of local inference.
  • Modal: Provided credits for CUDA verification and WER testing.
  • Blacksmith: Powered the project's CI/CD pipeline.
  • Hugging Face: Provided private storage for model hosting via the handy-computer org.

Sources

Related

  • Project
    handy-computer/transcribe.cpp

    transcribe.cpp is an MIT‑licensed C/C++ library that runs 60+ speech‑to‑text models (Whisper, Parakeet, Granite, Voxtral, etc.) via the GGUF format on Metal, Vulkan, CUDA, HIP/ROCm, or a tinyBLAS‑accelerated CPU path. It provides a CLI, a quantization tool, conversion scripts from NVIDIA NeMo, and official Python, TypeScript, Rust, and Swift bindings. Models are hosted on Hugging Face and are numerically verified with WER tests.

  • Dispatch
    Hugging Face Inference Endpoints: Fast Whisper Transcriptions

    Hugging Face has introduced a new OpenAI Whisper deployment option on Inference Endpoints that delivers up to 8x performance improvements in transcription speed without sacrificing accuracy.

  • Project
    ggml-org/whisper.cpp

    A high-performance C/C++ port of OpenAI's Whisper model for efficient, offline, on-device automatic speech recognition across diverse hardware.

  • Project
    CrispStrobe/CrispASR

    A unified C++ speech engine that provides a single binary for 119 ASR and TTS backends with zero Python dependencies.

  • Dispatch
    llama.cpp and llama.app: Local LLM Inference and Ecosystem

    llama.cpp and its official home llama.app provide high-performance, hardware-agnostic local LLM inference, now featuring a simplified 'llama serve' command and integration with local coding agents like Pi.