k2-fsa/sherpa-onnx

Speech-to-text, text-to-speech, speaker diarization, speech enhancement, source separation, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, HarmonyOS, Raspberry Pi, RISC-V, RK NPU, Axera NPU, Ascend NPU, x86_64 servers, websocket server/client, support 12 programming languages

What it solves

Sherpa-onnx provides a highly portable, local-first approach to speech and audio processing. It eliminates the need for cloud-based APIs by allowing developers to run complex audio AI models—such as speech-to-text, text-to-speech, and speaker identification—directly on a wide variety of hardware platforms and operating systems.

How it works

The project leverages the ONNX Runtime to execute pre-trained models locally. It provides a comprehensive set of APIs across numerous programming languages (including C++, Python, Go, Rust, and Swift) and supports a vast array of architectures (x86, ARM, RISC-V) and specialized hardware accelerators like NPUs (Rockchip, Qualcomm, Ascend, Intel OpenVINO).

Who it’s for

Developers building audio-centric applications that require low latency, privacy, or offline capability across mobile (Android, iOS), desktop (Windows, macOS, Linux), and embedded systems (Raspberry Pi, NVIDIA Jetson).

Highlights

  • Extensive Audio Suite: Supports ASR (streaming and non-streaming), TTS, speaker diarization, VAD, keyword spotting, and source separation.
  • Massive Platform Support: Compatible with almost every major OS and architecture, including WebAssembly and HarmonyOS.
  • Broad Language API: Offers bindings for 12+ programming languages, making it easy to integrate into existing codebases.
  • NPU Acceleration: Native support for various NPUs to improve performance on edge devices.

Related

  • Project
    istupakov/onnx-asr

    A lightweight Python package for Automatic Speech Recognition using ONNX models, eliminating the need for PyTorch or Transformers for fast deployment on edge and server hardware.

  • Project
    fxy2311-youyou/expression-trainer

    A local desktop application that uses offline speech recognition and AI to help users improve their oral expression by identifying filler words and vague language.

  • Project
    goodroot/hyprwhspr

    A native Linux speech-to-text tool providing fast, private, and system-wide dictation using local or cloud-based AI models.

  • Project
    0xShug0/audio.cpp

    A high-performance C++ audio inference framework built on ggml that provides a portable, native runtime for running a wide variety of local audio AI models.

  • Project
    thewh1teagle/kokoro-onnx

    A lightweight ONNX Runtime implementation of Kokoro-TTS that enables fast, multi-language text-to-speech generation on CPU and GPU.