0xShug0/audio.cpp

What it solves

audio.cpp is a high-performance C++ audio inference framework designed to eliminate the complexity of managing multiple Python environments and dependencies when running modern local audio models. It provides a shared, portable native runtime that allows users to run a wide variety of audio tasks—such as text-to-speech (TTS), automatic speech recognition (ASR), and voice conversion—across different hardware (NVIDIA, AMD, Apple Silicon, and CPU) without needing a Python-heavy stack.

How it works

Built on top of ggml, the framework provides a unified C++ surface for executing audio models. It supports a broad range of model families (over 60) and variants, utilizing GGUF loading and quantization (e.g., Q8) to reduce VRAM usage and increase inference speed. The framework includes a native WebUI, a CLI, and an API server, and it supports various backends including CUDA, HIP/ROCm, Vulkan, Metal, and CPU.

Who it’s for

Developers and users who want to run high-performance, local audio AI models with minimal setup overhead, and those looking for a portable, efficient deployment path for production-style audio tasks.

Highlights

• Extensive Model Support: Supports over 60 model families covering TTS, ASR, voice cloning, voice conversion, music generation, and source separation.
• High Performance: Significant speedups over Python reference paths, particularly on CUDA, with some models running up to 200x real-time.
• GGUF Integration: Broad support for GGUF loading and quantization to optimize memory and speed.
• Multi-Platform Portability: Native support for Windows, Linux, and macOS with multiple GPU backends.
• Integrated Tooling: Includes a native WebUI, an "Arena" for side-by-side model comparison, and built-in audio utilities for denoising and resampling.