Blaizzy/mlx-audio

A text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) library built on Apple's MLX framework, providing efficient speech analysis on Apple Silicon.

What it solves

MLX-Audio provides a unified, high-performance library for running advanced audio AI models—including text-to-speech (TTS), speech-to-text (STT), speech-to-speech (STS), and music generation—directly on Apple Silicon hardware.

How it works

It leverages Apple's MLX framework to optimize model inference for M-series chips. The library supports a wide range of model architectures, offering features like quantization (3-bit to 8-bit) to reduce memory usage, streaming audio output, and voice cloning. Users can interact with these models through a command-line interface, a Python API, or a built-in web server that includes 3D audio visualization.

Who it’s for

This project is for developers and researchers working on Apple Silicon who need to integrate or experiment with state-of-the-art audio models without relying on cloud-based APIs. It is also suitable for iOS/macOS developers looking to incorporate these capabilities into native applications via the provided Swift package.

Highlights

  • Optimized for Apple Silicon with support for various quantization levels (3-bit, 4-bit, 6-bit, 8-bit).
  • Extensive model support including Whisper, Kokoro, Qwen3-TTS, and many others.
  • Features include voice cloning, speech-to-speech interaction, and music generation.
  • Provides both a Python API and an OpenAI-compatible REST API for easy integration.
  • Includes an interactive web interface with real-time audio visualization.

Related

  • Project
    Blaizzy/mlx-audio-swift

    A modular Swift SDK for audio processing on Apple Silicon, providing a unified API for TTS, STT, VAD, and audio codecs using the MLX framework.

  • Project
    jrz97619761/test-model-thing

    Test‑Model‑Thing (TMT) is a small, MLX‑based language model (≈4.5 M parameters) that experiments with latent‑space prediction, recurrent trace units, byte‑level I/O, continuous streaming, and test‑time training. It runs on Apple Silicon (and Linux) and can be trained or used for chat via a simple CLI. The repo provides training, inference, and CoLA benchmarking scripts, but no pre‑trained weights; users must train their own model.

  • Project
    soniqo/speech-swift

    Speech Swift is an open‑source Swift package that provides on‑device speech AI for Apple Silicon. It bundles many modern ASR, TTS, translation, and voice‑agent models (e.g., Qwen3‑ASR, Whisper‑ASR, Omnilingual ASR, Qwen3‑TTS, CosyVoice, PersonaPlex) built on MLX and Core ML, so you can run inference locally with no cloud calls. Each model is a separate Swift‑Package‑Manager product, making it easy to add only what you need. The repo includes quick‑start code, SwiftUI UI helpers, Homebrew install, and extensive documentation.

  • Project
    senstella/parakeet-mlx

    Parakeet MLX is a Python package that runs Nvidia’s Parakeet speech‑to‑text models on Apple‑silicon Macs via the MLX framework. It offers a CLI and a Python API for transcribing audio files (or live streams) with timestamps, supporting greedy or beam decoding, chunking, local‑attention, and multiple output formats.

  • Project
    KoljaB/RealtimeTTS

    A Python library that provides a unified, low-latency interface for streaming text-to-speech, designed to integrate seamlessly with LLM token streams and various local or cloud-based audio engines.