argmaxinc/argmax-oss-swift

On-device Speech AI for Apple Silicon

What it solves

Argmax Open-Source SDK provides a set of turn-key frameworks for running AI audio models entirely on-device for Apple platforms (macOS and iOS). It eliminates the need for cloud-based APIs for speech-to-text, text-to-speech, and speaker identification, ensuring privacy and reducing latency.

How it works

The SDK is built on Core ML and consists of three specialized "kits":

  • WhisperKit: Implements OpenAI's Whisper for speech-to-text transcription and translation. It includes a local server that mimics the OpenAI Audio API for easy integration with existing clients.
  • TTSKit: Uses Qwen3-TTS models to convert text to speech with support for multiple languages, custom voices, and real-time streaming playback.
  • SpeakerKit: Utilizes Pyannote for speaker diarization, allowing the system to identify "who spoke when" in an audio recording.

Who it’s for

Developers building for macOS and iOS who want to integrate high-performance audio AI capabilities—like transcription, voice synthesis, and speaker identification—directly into their apps without relying on external servers.

Highlights

  • On-Device Inference: All processing happens locally on Apple silicon via Core ML.
  • OpenAI API Compatibility: Includes a local server that allows developers to use standard OpenAI SDK clients for local transcription.
  • Streaming Capabilities: Supports real-time streaming playback for TTS and incremental loading for large audio files in WhisperKit to manage memory.
  • Multilingual Support: TTSKit supports 10 languages and multiple built-in voices; WhisperKit supports various model sizes for different accuracy/speed trade-offs.

Related

  • Project
    FluidInference/FluidAudio

    A Swift SDK for local, low-latency audio AI on Apple devices, providing transcription, text-to-speech, and speaker diarization offloaded to the Apple Neural Engine.

  • Project
    Blaizzy/mlx-audio-swift

    A modular Swift SDK for audio processing on Apple Silicon that provides easy integration of TTS, STT, and speaker diarization models using the MLX framework.

  • Project
    TypeWhisper/typewhisper-mac

    A system-wide speech-to-text and AI text processing utility for macOS that supports both local on-device models and cloud APIs for transcription and transformation.

  • Project
    TheStageAI/TheWhisper

    A high-performance speech-to-text solution based on fine-tuned Whisper models, optimized for low-latency streaming and on-device inference on NVIDIA GPUs and Apple Silicon.

  • Project
    soniqo/speech-swift

    Speech Swift is an open‑source Swift package that provides dozens of on‑device speech AI models (ASR, TTS, translation, voice agents, enhancement, etc.) for Apple Silicon. It runs via Core ML and the MLX framework, supports hundreds of languages, offers multiple quantised sizes, and can be added to a project with a single Swift‑Package‑Manager dependency. The repo includes extensive docs, pre‑trained models, and example code for transcription, streaming dictation, and UI components, all without any cloud calls.