sooftware/conformer

[Unofficial] PyTorch implementation of "Conformer: Convolution-augmented Transformer for Speech Recognition" (INTERSPEECH 2020)

What it solves

It addresses the limitation of using either Transformers or CNNs alone for speech recognition. While Transformers are excellent at capturing global interactions, CNNs are better at extracting local features. This project provides a PyTorch implementation of the Conformer architecture to model both types of dependencies in audio sequences efficiently.

How it works

The project implements the Conformer model, which is a convolution-augmented Transformer. By combining the strengths of CNNs and Transformers, the model can capture both local and global dependencies within an audio sequence in a parameter-efficient manner.

Who it’s for

This is for developers and researchers working on automatic speech recognition (ASR) who want to integrate the Conformer architecture into their PyTorch-based audio processing pipelines.

Highlights

  • PyTorch implementation of the Conformer architecture.
  • Combines CNNs and Transformers for improved accuracy in speech recognition.
  • Parameter-efficient modeling of audio sequences.
  • Compatible with CTC Loss for training.

Related

  • Project
    k2-fsa/sherpa

    An open-source speech-to-text inference framework using PyTorch for deploying pre-trained transducer and CTC-based models.

  • Project
    pytorch/audio

    A PyTorch-based audio library that provides tools for processing audio data for machine learning, featuring GPU acceleration and specialized audio transforms.

  • Project
    henry9zhr/QANet-pytorch

    A PyTorch implementation of QANet, a convolutional neural network architecture for question answering tasks.

  • Project
    nschaetti/EchoTorch

    A PyTorch-based module for implementing and testing Echo State Networks and Reservoir Computing, designed for research purposes and integration into deep architectures.

  • Project
    nicklashansen/dreamer4

    A PyTorch implementation of the Dreamer 4 world model designed for multi-task continuous control, featuring a causal tokenizer and an interactive dynamics model.