MahmoudAshraf97/ctc-forced-aligner

Text to speech alignment using CTC forced alignment

What it solves

This project provides a way to perform forced alignment between audio files and their corresponding text transcripts. This process identifies the exact start and end timestamps for specific segments of text (such as words, characters, or sentences) as they are spoken in an audio recording, which is essential for creating high-quality speech corpuses.

How it works

The tool leverages pretrained Connectionist Temporal Classification (CTC) models from Hugging Face, specifically Wav2Vec2, HuBERT, and MMS models. It processes audio waveforms and text transcripts, using these models to map the spoken sounds to the written text, outputting the results in a structured JSON format with precise timestamps.

Who it’s for

It is designed for researchers and developers working with speech-to-text data, linguists, and anyone needing to synchronize audio and text for subtitles, speech corpus creation, or audio analysis.

Highlights

  • Memory Efficiency: Uses significantly less memory (at least 5X less) than the TorchAudio forced alignment API.
  • Broad Language Support: Supports over 1,100 languages, including English, Arabic, Russian, and German.
  • Flexible Granularity: Allows alignment at the sentence, word, or character level.
  • GPU Acceleration: Supports CUDA for faster inference.
  • Structured Output: Generates JSON files containing the aligned text and precise timestamps for each segment.

Related

  • Project
    readbeyond/aeneas

    A Python/C library and toolset for forced alignment, automatically synchronizing audio narrations with their corresponding text fragments.

  • Project
    strob/gentle

    A robust forced-aligner built on Kaldi that aligns speech audio with text transcripts to provide precise word-level timing.

  • Project
    speechbrain/speechbrain

    An open-source PyTorch toolkit for Conversational AI that provides recipes and tools for speech and text processing, including speech recognition and synthesis.

  • Project
    MontrealCorpusTools/Montreal-Forced-Aligner

    A command-line utility for performing forced alignment of speech datasets, synchronizing audio recordings with their text transcripts using Kaldi.

  • Project
    snakers4/silero-models

    A collection of fast, end-to-end text-to-speech models supporting a wide variety of languages, designed for efficient execution on CPU and GPU.