FluidInference/FluidAudio

Frontier CoreML audio models in your apps — text-to-speech, speech-to-text, voice activity detection, and speaker diarization. In Swift, powered by SOTA open source.

What it solves

FluidAudio provides on‑device, low‑latency speech capabilities for Apple hardware. It lets apps transcribe speech, synthesize voice, detect when someone is speaking, separate speakers, and clean up audio—all without sending data to the cloud.

How it works

The SDK ships Swift bindings to CoreML models that have been converted and optimized for Apple’s Neural Engine (ANE). Models cover automatic speech recognition (ASR) for dozens of languages, inverse text normalization, text‑to‑speech, voice activity detection, speaker diarization, speaker embedding extraction, and acoustic echo/noise suppression. Developers add the desired features with a few Swift calls; the heavy lifting runs on the ANE, keeping CPU use low and preserving privacy.

Who it’s for

  • iOS/macOS developers building privacy‑first voice assistants, dictation tools, meeting‑note apps, or any app that needs fast, offline speech processing.
  • Companies that want to embed speech AI in their products without relying on cloud APIs.
  • Researchers interested in running state‑of‑the‑art audio models on Apple Silicon.

Highlights

  • ASR models (Parakeet, SenseVoice, Paraformer) support 25+ European languages, Japanese, Mandarin, and streaming real‑time transcription.
  • Built‑in inverse text normalization to turn spoken numbers into digits.
  • Multilingual TTS models (Kokoro, PocketTTS, Chatterbox) with SSML control and voice cloning.
  • Real‑time speaker diarization and embedding extraction for speaker‑aware applications.
  • Voice activity detection and echo/noise suppression running at ~16 ms latency.
  • All models run on the ANE for minimal power draw and fast inference.
  • Open‑source model weights hosted on Hugging Face under permissive licenses.

Related

  • Project
    argmaxinc/argmax-oss-swift

    A collection of on-device inference frameworks for Apple platforms providing speech-to-text, text-to-speech, and speaker diarization using Core ML.

  • Project
    altic-dev/FluidVoice

    FluidVoice is a macOS‑only, open‑source dictation app that provides real‑time transcription using a range of local speech models and an optional on‑device AI layer (Fluid Intelligence) for smart formatting. It supports command‑voice control, per‑app prompts, global hotkeys, and privacy‑first operation, and can be installed via Homebrew or built from source.

  • Project
    soniqo/speech-swift

    Speech Swift is an open‑source Swift package that provides on‑device speech AI for Apple Silicon. It bundles many modern ASR, TTS, translation, and voice‑agent models (e.g., Qwen3‑ASR, Whisper‑ASR, Omnilingual ASR, Qwen3‑TTS, CosyVoice, PersonaPlex) built on MLX and Core ML, so you can run inference locally with no cloud calls. Each model is a separate Swift‑Package‑Manager product, making it easy to add only what you need. The repo includes quick‑start code, SwiftUI UI helpers, Homebrew install, and extensive documentation.

  • Project
    FluidInference/FluidUse

    FluidUse is a Swift library for on‑device form‑filling and typed‑decision making on macOS. It reads UI fields via Accessibility, queries tiny Core ML models (form‑filler, Laya, GLiNER2, Kev, Clef‑Vision, Intern‑Decision, Decision 2.0, Vela 2.0, Short‑Reply) and types or acts on the answers, all locally with sub‑millisecond latency on Apple Silicon.

  • Project
    homelab-00/TranscriptionSuite

    A free, open-source transcription app that converts speech to text locally on your computer using various AI models for privacy and speed.