openspeech-team/openspeech

Open-Source Toolkit for End-to-End Speech Recognition leveraging PyTorch-Lightning and Hydra.

What it solves

OpenSpeech is a framework designed to simplify the creation of end-to-end (E2E) automatic speech recognition (ASR) systems. It replaces traditional "hybrid" ASR systems—which require separate, complex training for acoustic, language, and pronunciation models—with a single integrated approach that reduces training and decoding time.

How it works

Built on PyTorch-Lightning and Hydra, OpenSpeech provides a hardware-agnostic environment for training ASR models. It allows users to leverage advanced features like mixed-precision training, multi-node training, and TPU support without complex engineering. The framework includes reference implementations of over 20 ASR modeling papers, support for common audio features (Spectrogram, Mel-Spectrogram, Filter-Bank, MFCC), and various data augmentation techniques such as SpecAugment.

Who it’s for

It is designed for researchers, educators, and practitioners who want to experiment with famous ASR models or build custom speech recognizers using provided modules and recipes for English, Chinese, and Korean languages.

Highlights

  • Extensive Model Library: Supports 20+ ASR architectures including Conformer, Transformer, Jasper, QuartzNet, and DeepSpeech2.
  • Hardware Agnostic: Seamlessly supports GPU and TPU training via PyTorch-Lightning.
  • Language Recipes: Includes pre-configured recipes for LibriSpeech (English), AISHELL-1 (Chinese), and KsponSpeech (Korean).
  • Flexible Configuration: Uses Hydra for hierarchical, dynamic configuration management of training hyperparameters.

Related

  • Project
    espnet/espnet

    An end-to-end speech processing toolkit that provides a unified framework for ASR, TTS, speech translation, and enhancement using PyTorch.

  • Project
    flashlight/wav2letter

    wav2letter++ is an end-to-end automatic speech recognition framework that provides recipes and pre-trained models to implement state-of-the-art speech-to-text architectures.

  • Project
    yeyupiaoling/MASR

    A PyTorch-based automatic speech recognition framework that supports streaming and non-streaming inference across multiple model architectures and languages.

  • Project
    istupakov/onnx-asr

    A lightweight Python package for Automatic Speech Recognition using ONNX models, eliminating the need for PyTorch or Transformers for fast deployment on edge and server hardware.

  • Project
    CrispStrobe/CrispASR

    A unified C++ speech engine providing a single binary for 54 ASR and 52 TTS backends with zero Python dependencies.