k2-fsa/sherpa-onnx

Speech-to-text, text-to-speech, speaker diarization, speech enhancement, source separation, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, HarmonyOS, Raspberry Pi, RISC-V, RK NPU, Axera NPU, Ascend NPU, x86_64 servers, websocket server/client, support 12 programming languages

What it solves

This project provides a comprehensive, high-performance toolkit for running speech and audio processing models locally on a wide variety of devices. It eliminates the need for cloud-based APIs by enabling tasks like speech-to-text, text-to-speech, and speaker identification to run directly on consumer hardware, mobile devices, and embedded systems.

How it works

It leverages the ONNX Runtime to execute pre-trained audio models across diverse hardware architectures. By supporting a vast array of programming languages and operating systems, it allows developers to integrate sophisticated audio AI features into applications ranging from desktop software and mobile apps to embedded systems and web browsers via WebAssembly.

Who it’s for

This is for developers and engineers who need to deploy audio AI models (such as ASR, TTS, or VAD) in resource-constrained environments or privacy-sensitive applications where data must remain on-device. It is particularly useful for those targeting cross-platform compatibility, including Android, iOS, Windows, Linux, and specialized NPU hardware.

Highlights

  • Supports a wide range of audio tasks including speech recognition (streaming and non-streaming), speech synthesis, speaker diarization, and keyword spotting.
  • Highly portable with support for numerous programming languages (C++, Python, Java, Go, Rust, Swift, etc.) and platforms (Android, iOS, WebAssembly, etc.).
  • Optimized for diverse hardware, including support for various NPUs (Rockchip, Qualcomm, Intel, etc.) and edge devices like Raspberry Pi and NVIDIA Jetson.
  • Provides extensive pre-trained models and ready-to-use demo applications for mobile and desktop environments.

Related

  • Project
    istupakov/onnx-asr

    A lightweight Python package for running Automatic Speech Recognition models using ONNX, designed to be fast and dependency-minimal without requiring PyTorch or Transformers.

  • Project
    fxy2311-youyou/expression-trainer

    A local desktop application that uses offline speech recognition and AI to help users improve their oral expression by identifying filler words and vague language.

  • Project
    thewh1teagle/kokoro-onnx

    A lightweight ONNX Runtime implementation of Kokoro-TTS that enables fast, multi-language text-to-speech generation on CPU and GPU.

  • Project
    Sharrnah/whispering-ui

    A native desktop application for Windows and Linux that provides a local, offline AI interface for real-time audio transcription, translation, and text-to-speech.

  • Project
    Uberi/speech_recognition

    A Python library that provides a unified interface for speech recognition, supporting multiple online and offline engines like OpenAI Whisper, Google Speech, and Vosk.