PaddlePaddle/PaddleSpeech

Easy-to-use Speech Toolkit including Self-Supervised Learning model, SOTA/Streaming ASR with punctuation, Streaming TTS with text frontend, Speaker Verification System, End-to-End Speech Translation and Keyword Spotting. Won NAACL2022 Best Demo Award.

What it solves

PaddleSpeech is an open-source toolkit designed to simplify the implementation of critical speech and audio tasks. It provides a unified platform for developers and researchers to perform speech recognition, text-to-speech synthesis, and other audio processing tasks without having to build complex pipelines from scratch.

How it works

Built on the PaddlePaddle platform, the toolkit integrates state-of-the-art models and mainstream datasets (such as LibriSpeech and AIShell). It offers a flexible architecture that supports the entire pipeline from training, inference, and testing to final deployment. Users can interact with the toolkit via a Command Line Interface (CLI), a dedicated Server, or a Streaming Server for production-ready real-time applications.

Who it’s for

It is intended for both industrial application developers who need production-ready speech systems and academic researchers focusing on audio and speech technology.

Highlights

  • Comprehensive Audio Tasks: Supports Automatic Speech Recognition (ASR), Text-to-Speech (TTS), Speaker Verification, Keyword Spotting, Audio Classification, and Speech Translation.
  • Production Ready: Includes streaming ASR and TTS systems for real-time use cases.
  • Chinese Language Optimization: Features a rule-based Chinese frontend for text normalization and Grapheme-to-Phoneme (G2P) conversion.
  • Versatile Deployment: Provides CLI, Server, and Streaming Server options, with support for ONNX format and C++ high-performance deployment.
  • Advanced Capabilities: Includes punctuation restoration, voice cloning, and singing voice synthesis (SVS).

Related

  • Project
    brailcom/speechd

    A device-independent interface for speech synthesis that provides a unified API for various Text-to-Speech engines to support accessibility software.

  • Project
    kadirnar/whisper-plus

    A comprehensive audio and video processing toolkit that extends OpenAI's Whisper for transcription, speaker diarization, summarization, and RAG-based video chatting.

  • Project
    PaddlePaddle/PaddleX

    A low-code AI development tool based on PaddlePaddle that provides pre-trained model pipelines for rapid training, inference, and deployment across diverse hardware.

  • Project
    wenet-e2e/wenet

    A production-oriented end-to-end speech recognition toolkit that provides full-stack solutions for high-accuracy transcription of English and Chinese audio.

  • Project
    jianchang512/pyvideotrans

    An open-source video translation tool that automates speech recognition, subtitle translation, and AI dubbing with support for voice cloning and multi-role synthesis.