QwenAudio/Fun-ASR

Open-source LLM-based ASR model family for Chinese, dialect, accent, and multilingual speech, with FunASR, vLLM, streaming, and llama.cpp runtimes.

What it solves

Fun-ASR provides high-precision, end-to-end speech recognition (ASR) designed to handle challenging real-world audio. It specifically addresses the difficulty of transcribing audio with heavy background noise, far-field sound pickup (such as in conference rooms or vehicles), and the complexity of diverse Chinese dialects and regional accents.

How it works

The project offers a family of models, most notably the 800M parameter Nano checkpoints. These models are trained on massive datasets—up to tens of millions of hours of speech—to ensure robustness. The system can be deployed in several ways:

  • PyTorch Native: Standard inference for general use.
  • vLLM Engine: A high-throughput path that is 3-5x faster for batch processing and supports production-grade WebSocket streaming for real-time subtitles.
  • llama.cpp/GGUF: A self-contained binary version that allows the model to run on CPUs or edge devices without requiring a Python runtime.
  • Composed Pipelines: While the Nano models don't natively handle speaker labels or timestamps, they can be combined with separate models (like FSMN-VAD for voice activity detection and CAM++ for speaker diarization) to produce labeled transcripts.

Who it’s for

This tool is for developers and engineers building speech-to-text applications that require high accuracy in noisy environments, support for a wide array of East and Southeast Asian languages, or the ability to deploy on resource-constrained edge hardware.

Highlights

  • Extensive Language Support: Includes a multilingual checkpoint supporting 31 languages and a specialized Nano checkpoint for Chinese, English, Japanese, and 7 major Chinese dialects.
  • Noise Robustness: Optimized for far-field and high-noise scenarios, reaching up to 93% recognition accuracy.
  • Specialized Recognition: Capable of recognizing lyrics and rap speech even with music background interference.
  • High Performance: vLLM integration enables massive speedups (up to 16x faster than PyTorch native) for batch transcription.

Related

  • Project
    modelscope/FunASR

    An industrial speech recognition toolkit for offline, streaming, and edge deployment, providing high-performance ASR, VAD, and speaker diarization pipelines.

  • Project
    FireRedTeam/FireRedASR2S

    An industrial-grade all-in-one ASR system that combines speech recognition, voice activity detection, language identification, and punctuation prediction, with specialized support for Chinese dialects.

  • Project
    istupakov/onnx-asr

    A lightweight Python package for Automatic Speech Recognition using ONNX models, eliminating the need for PyTorch or Transformers for fast deployment on edge and server hardware.

  • Project
    AudarAI/Audar-ASR-V1

    A family of Arabic-first generative speech recognition models that provide state-of-the-art transcription for dialectal Arabic and code-switched speech.

  • Project
    NVIDIA-NeMo/Speech

    A PyTorch-based toolkit for developing and deploying professional speech AI models, including ASR, TTS, and Speech LLMs.