soniqo/speech-swift

AI speech toolkit for Apple Silicon — ASR, TTS, speech-to-speech, VAD, and diarization powered by MLX and CoreML

What it solves

Speech Swift provides a comprehensive suite of on-device AI speech models for Apple Silicon (Mac and iOS). It eliminates the need for cloud APIs and external data transmission by enabling local speech recognition, synthesis, and understanding directly on the device's hardware.

How it works

The library is powered by MLX Swift and CoreML, leveraging the Apple Neural Engine (ANE) and GPU for high-performance inference. It is structured as a collection of Swift Package Manager (SPM) modules, allowing developers to import only the specific models they need (e.g., ParakeetStreamingASR or Qwen3TTS) to keep application footprints small.

Who it’s for

It is designed for iOS and macOS developers building voice agents, transcription tools, and speech-generation applications that require low latency, privacy-first local execution, and native Apple hardware acceleration.

Highlights

  • Broad Capability Set: Supports automatic speech recognition (ASR), text-to-speech (TTS), speech-to-speech translation, speaker diarization, and voice activity detection (VAD).
  • Extensive Model Library: Includes implementations of state-of-the-art models like Whisper, Qwen3, Nemotron, and Kokoro, with some supporting over 1,600 languages.
  • High Performance: Optimized for Apple Silicon with benchmarks showing low Real-Time Factors (RTF) on devices like the iPhone 16 Pro.
  • Privacy-Centric: All processing happens locally on-device, requiring no API keys or cloud connectivity.
  • Developer-Friendly: Provides a SpeechUI module with SwiftUI views for rapid integration of transcription interfaces.