QwenAudio/SenseVoice

Open-source SenseVoiceSmall model for Mandarin, Cantonese, English, Japanese, and Korean ASR, language ID, emotion recognition, and audio event detection.

What it solves

SenseVoice is a speech foundation model designed to handle multiple audio understanding tasks simultaneously. It addresses the need for high-accuracy, low-latency speech recognition across multiple languages while also capturing non-verbal cues like emotion and environmental sounds, which are often lost in standard ASR systems.

How it works

SenseVoice uses a non-autoregressive end-to-end framework to achieve low-latency inference. The released SenseVoiceSmall checkpoint supports automatic speech recognition (ASR), spoken language identification (LID), speech emotion recognition (SER), and audio event detection (AED). It can be deployed as a Python-based service via FastAPI, exported to ONNX or Libtorch for optimized performance, or run as a self-contained binary on CPU/edge devices using llama.cpp/GGUF.

Who it’s for

Developers and researchers building voice-enabled applications that require multilingual support (specifically Mandarin, Cantonese, English, Japanese, and Korean) and the ability to detect emotional states or specific audio events (like laughter or coughing) in real-time.

Highlights

  • Multitask Capability: Performs ASR, language identification, emotion recognition, and audio event detection in one model.
  • High Efficiency: Non-autoregressive architecture allows it to run significantly faster than Whisper-Small and Whisper-Large.
  • Rich Transcription: Detects common human-computer interaction events (e.g., applause, crying, sneezing) and emotional tones.
  • Flexible Deployment: Supports GPU, CPU, and edge deployment via GGUF, with no Python runtime required for the binary version.
  • Finetuning Support: Includes scripts and strategies to adapt the model to specific business scenarios or long-tail samples.

Related

  • Project
    QwenAudio/CosyVoice

    CosyVoice is an advanced, LLM-based text-to-speech system designed for high-quality, zero-shot multilingual speech synthesis and voice cloning.

  • Project
    espressif/esp-sr

    ESP‑SR is Espressif’s on‑device speech‑recognition framework for ESP32 MCUs. It bundles a tiny audio front‑end, wake‑word detection (WakeNet), voice‑activity detection (VADNet), offline command spotting (MultiNet), and speech synthesis, all optimized for low‑memory, low‑power chips. The library is modular, supports multiple languages, and lets developers add voice control to IoT devices without needing cloud services.

  • Project
    QwenAudio/Fun-ASR

    A family of high-precision end-to-end speech recognition models optimized for noisy environments, diverse Chinese dialects, and high-throughput production deployment.

  • Project
    FireRedTeam/FireRedASR2S

    An industrial-grade all-in-one ASR system that combines speech recognition, voice activity detection, language identification, and punctuation prediction, with specialized support for Chinese dialects.

  • Project
    TensorSpeech/TensorFlowASR

    A TensorFlow-based framework for Automatic Speech Recognition that implements various ASR architectures and supports TFLite conversion for efficient deployment.