soniqo/speech-swift

What is Speech Swift?

Speech Swift is an open‑source Swift package that brings a whole suite of modern AI speech models to Apple‑silicon devices (Mac, iPhone, iPad). All inference runs locally – no cloud service, no API keys, and your audio never leaves the device. The library is built on two Apple‑friendly runtimes:

• MLX – a fast GPU‑accelerated framework for running large language‑model‑style networks on macOS.
• Core ML – Apple’s on‑device inference engine that can run on the Neural Engine (ANE) for ultra‑low latency.

The package ships each model as an independent Swift‑Package‑Manager product, so you only pull in the models you actually need.

---

Main capabilities (grouped by task)

Category	Example models (size)	What they do
Speech‑to‑Text (ASR)	Qwen3‑ASR (0.6‑1.7 B), Whisper‑ASR (Large‑v3 Turbo), Omnilingual ASR (up to 7 B, 1 672 languages)	Convert audio to text, many support streaming, punctuation, speaker diarisation, and multilingual output.
Alignment	Qwen3‑ForcedAligner (0.6 B)	Takes audio + transcript and returns word‑level timestamps.
Text‑to‑Speech (TTS)	Qwen3‑TTS (0.6‑1.7 B), CosyVoice 3 (0.5 B), Kokoro 82 M, Supertonic 3 (99 M)	High‑quality, streaming synthesis; many models support zero‑shot voice cloning, emotion tags, and multi‑speaker output.
LLM & Translation	Qwen3Chat (0.8‑4 B), FunctionGemma (270 M), MADLAD‑400 (3 B, 400+ languages)	On‑device chat, tool‑calling, and many‑to‑many text translation.
Speech‑to‑Speech / Voice Agents	PersonaPlex (7 B), VoiceChat 11 B, Hibiki Zero‑3 B	Full‑duplex audio‑in → audio‑out pipelines, enabling conversational agents that can listen and speak continuously.
Audio enhancement	DeepFilterNet3 (2.1 M), LocalVQE AEC, Source‑separation (HTDemucs + Open‑Unmix)	Real‑time noise suppression, echo cancellation, and music‑stem separation.
Utility	Wake‑word spotting, VAD, speaker diarisation, speaker embeddings	Build robust voice‑activated interfaces and identify who is speaking.

---

How to get started (quick example)

import ParakeetStreamingASR   // pull only the streaming ASR product

let model = try await ParakeetStreamingASRModel.fromPretrained()
let text  = try model.transcribeAudio(audioSamples, sampleRate: 16_000)

The same model also offers a transcribeStream async sequence for live dictation.

A minimal SwiftUI dictation view is provided by the optional SpeechUI product –‑ just import it and drop the TranscriptionView into your UI.

---

Who is it for?

• iOS/macOS developers who want on‑device speech capabilities without paying for cloud APIs.
• Researchers looking for a convenient way to benchmark large speech models on Apple hardware.
• Product teams building voice assistants, transcription tools, or multimodal AI agents that must respect user privacy.

---

Where to find more

• Full docs & model catalogue – https://soniqo.audio
• Pre‑trained model files – hosted on Hugging Face under the aufklarer organization.
• Community – Discord server (invite in README) and a series of blog posts that walk through performance comparisons, backend selection (MLX vs Core ML), and end‑to‑end demos.
• Installation – brew install speech also works via Homebrew.

---

TL;DR

Speech Swift is a comprehensive, on‑device Swift library that lets you run state‑of‑the‑art speech‑to‑text, text‑to‑speech, translation, and voice‑agent models locally on Apple Silicon, using either MLX or Core ML. It’s modular, well‑documented, and backed by a growing ecosystem of open‑source projects that already depend on it.