deeeed/audiolab

Cross-platform audio processing SDK for React Native — streaming, recording, analysis, visualization, and on-device inference

What it solves

Audiolab is a comprehensive toolkit for integrating advanced audio processing into React Native and Expo apps. It solves the difficulty of implementing high-performance, on-device audio recording, analysis, and speech processing across iOS, Android, and the web.

How it works

It is organized as a monorepo containing specialized packages for different audio tasks:

  • Core Processing: @siteed/audio-studio handles the fundamental recording and analysis functions.
  • Visualization: @siteed/audio-ui provides components for audio visualization using Skia.
  • Feature Extraction: @siteed/react-native-essentia uses the Essentia library to extract audio features.
  • On-Device Speech: @siteed/sherpa-onnx.rn and @siteed/moonshine.rn provide on-device speech-to-text (STT), text-to-speech (TTS), and intent recognition.

Who it’s for

Developers building mobile and web applications with React Native and Expo who need professional-grade audio capabilities, such as transcription, voice assistants, and audio analysis tools.

Highlights

  • On-Device Processing: Supports on-device transcription and intent recognition, reducing latency and privacy concerns.
  • Cross-Platform: Unified API for iOS, Android, and web.
  • High Performance: Includes C++ based audio analysis via Essentia and zero-latency recording.
  • Full Ecosystem: Includes a comprehensive demo app (Audio Playground) to showcase all integrated features.

Related

  • Project
    jamsch/expo-speech-recognition

    A cross-platform React Native library that provides a unified API for speech-to-text functionality across iOS, Android, and the Web.

  • Project
    MTG/essentia

    An open-source C++ and Python library for audio analysis and music information retrieval, providing a wide array of signal processing algorithms and music descriptors.

  • Project
    soniqo/speech-swift

    A library of on-device AI speech models for Apple Silicon, providing local speech recognition, synthesis, and understanding for Mac and iOS apps.

  • Project
    Blaizzy/mlx-audio-swift

    A modular Swift SDK for audio processing on Apple Silicon, providing on-device support for text-to-speech, speech-to-text, and speaker diarization using the MLX framework.

  • Project
    kadirnar/whisper-plus

    A comprehensive audio and video processing toolkit that extends OpenAI's Whisper for transcription, speaker diarization, summarization, and RAG-based video chatting.