Blaizzy/mlx-audio

A text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) library built on Apple's MLX framework, providing efficient speech analysis on Apple Silicon.

What it solves

MLX-Audio provides a high-performance audio processing library specifically optimized for Apple Silicon (M series chips). It simplifies the deployment of complex audio AI models for text-to-speech (TTS), speech-to-text (STT), and speech-to-speech (STS) tasks, allowing users to run these models locally with high efficiency.

How it works

Built on Apple's MLX framework, the library integrates a wide variety of pre-trained models from the community. It supports multiple modalities including:

  • Text-to-Speech (TTS): Generating natural speech from text with support for voice cloning, multilingual output, and emotion control.
  • Speech-to-Text (STT): Transcribing audio to text, including word-level alignment, speaker diarization (identifying who is speaking), and streaming transcription.
  • Speech-to-Speech (STS): Handling tasks like source separation, speech enhancement, and noise suppression.
  • VAD/Diarization: Detecting voice activity and separating different speakers.

Who it’s for

  • Developers building audio-centric applications for macOS and iOS.
  • Researchers wanting to run state-of-the-art audio models on Apple hardware.
  • Users who need fast, local, and private audio processing without relying on cloud APIs.

Highlights

  • Apple Silicon Optimized: Fast inference tailored for M-series chips.
  • वाइन-compatible API: Includes an OpenAI-compatible REST API and a web interface with 3D audio visualization.
  • Extensive Model Support: Integrates dozens of models like Whisper, Kokoro, Qwen3-TTS, and VibeVoice-ASR.
  • C-level Integration: Provides a Swift package for native iOS/macOS integration.
  • Performance Tuning: Supports quantization (3-bit to 8-bit) to reduce memory usage and increase speed.