FireRedTeam/FireRedASR2S

A SOTA Industrial-Grade All-in-One ASR system with ASR, VAD, LID, and Punc modules. FireRedASR2 supports Chinese (Mandarin, 20+ dialects/accents), English, code-switching, and both speech and singing ASR. FireRedVAD supports speech/singing/music in 100+ langs. FireRedLID supports 100+ langs and 20+ zh dialects. FireRedPunc supports zh and en.

What it solves

FireRedASR2S is an industrial-grade, all-in-one automatic speech recognition (ASR) system designed to handle complex audio transcription tasks. It addresses the need for a unified pipeline that can detect voice activity, identify spoken languages (including numerous Chinese dialects), transcribe speech and singing, and automatically add punctuation to the resulting text.

How it works

The system integrates four specialized modules that can be used together as a complete pipeline or independently:

  • FireRedASR2: The core transcription engine. It offers two variants: an LLM-based version for state-of-the-art performance and end-to-end interaction, and an AED (Attention-based Encoder-Decoder) version that balances performance with computational efficiency.
  • FireRedVAD: A Voice Activity Detection module that identifies speech, singing, or music across 100+ languages. It supports both streaming and non-streaming modes.
  • FireRedLID: A Spoken Language Identification module that recognizes 100+ languages and over 20 Chinese dialects/accents.
  • FireRedPunc: A punctuation prediction module that adds appropriate punctuation to Chinese and English transcriptions.

Who it’s for

This system is intended for developers and engineers building industrial-grade speech-to-text applications, particularly those requiring high accuracy for Mandarin, various Chinese dialects, English, and code-switching scenarios.

Highlights

  • Comprehensive Dialect Support: High accuracy for Mandarin and 20+ Chinese dialects/accents (e.g., Cantonese, Sichuan, Shanghai).
  • All-in-One Pipeline: Combines VAD, LID, ASR, and Punctuation in a single system.
  • Flexible Architectures: Provides both LLM and AED model variants to choose between maximum accuracy and higher efficiency.
  • High Performance: Outperforms several industry benchmarks (such as Whisper and Fun-ASR) in CER and F1 scores.
  • Inference Acceleration: Supports TensorRT-LLM for significant speedups (up to 12.7x) and integrates with vLLM for high-performance serving.

Related

  • Project
    QwenAudio/Fun-ASR

    A family of high-precision end-to-end speech recognition models optimized for noisy environments, diverse Chinese dialects, and high-throughput production deployment.

  • Project
    modelscope/FunASR

    An industrial speech recognition toolkit for offline, streaming, and edge deployment, providing high-performance ASR, VAD, and speaker diarization pipelines.

  • Project
    QwenAudio/SenseVoice

    A multilingual speech foundation model that provides high-accuracy ASR, emotion recognition, and audio event detection with exceptionally low inference latency.

  • Dispatch
    Fine-Tuning Wav2Vec2-BERT for Low-Resource ASR

    Hugging Face demonstrates how to fine-tune Meta's Wav2Vec2-BERT model for Automatic Speech Recognition (ASR) in low-resource languages, achieving performance comparable to Whisper-large-v3 while being significantly faster and more resource-efficient.

  • Project
    yeyupiaoling/MASR

    A PyTorch-based automatic speech recognition framework that supports streaming and non-streaming inference across multiple model architectures and languages.