yeyupiaoling/MASR

What it solves

MASR (Magical Automatic Speech Recognition) is a PyTorch-based framework designed to make automatic speech recognition (ASR) simple and practical. It provides a unified system for converting spoken audio into text, supporting both real-time (streaming) and batch (non-streaming) processing across various languages and platforms.

How it works

The framework implements a complete ASR pipeline, from audio preprocessing to final text output:

• Models: It supports multiple architectures including Conformer, Squeezeformer, Efficient Conformer, and DeepSpeech2.
• Preprocessing: Audio is processed using methods like fbank and mfcc, utilizing the kaldi_native_fbank library for improved speed and cross-platform compatibility.
• Decoding: It employs various decoding strategies such as CTC greedy search, CTC prefix beam search, CTC beam search, and attention rescoring.
• Tokenization: It uses sentencepiece for tokenization, which simplifies the handling of multiple languages and enables mixed-language (e.g., Chinese and English) training.
• Data Augmentation: To improve robustness, it includes noise, reverberation, speed, volume, resampling, and shift augmentation, as well as SpecAugment and SpecSubAugment.

Who it’s for

MASR is intended for developers and researchers who need a deployable ASR system that can run on servers, Nvidia Jetson devices, and potentially mobile devices (Android).

Highlights

• Flexible Inference: Supports short audio, long audio, streaming, and speaker-diarization-based inference.
• Multi-language Support: Capable of handling Mandarin, English, Cantonese, and Uyghur, including mixed Chinese-English models.
• Cross-Platform: Compatible with Windows, Linux, and macOS.
• Comprehensive Tooling: Includes built-in scripts for data preparation, synthetic speech generation, model training, evaluation, and model export.