remsky/Kokoro-FastAPI

Kokoro‑FastAPI – Dockerized OpenAI‑compatible TTS service

What it is – A ready‑to‑run FastAPI server that wraps the Kokoro‑82M text‑to‑speech model (a 82 million‑parameter multilingual TTS model). The server mimics the OpenAI audio/speech endpoint, so existing OpenAI client libraries can be used unchanged. It is shipped as multi‑arch Docker images (CPU, NVIDIA‑CUDA, AMD‑ROCm, Apple‑Silicon) and can also be started directly with uv.

Key capabilities

• OpenAI‑compatible Speech API – POST /v1/audio/speech accepts the same JSON fields (model, voice, input, response_format, speed, …) and returns audio in mp3, wav, opus, flac, aac or raw PCM.
• Streaming – Both the OpenAI client and plain requests can receive audio chunk‑by‑chunk, enabling real‑time playback with low first‑token latency (≈300 ms on GPU, ≈1 s on modern M‑series CPU).
• Multilingual voices – English (US/GB), Spanish, French, Hindi, Italian, Japanese, Brazilian Portuguese, Mandarin Chinese.
• Voice mixing & aliases – Combine several voicepacks with weighted ratios (af_bella(2)+af_sky(1)) or define short aliases for complex mixes.
• Inline SSML‑style tags – [voice:…], [pause:1.5s], [rate:1.5], and IPA pronunciation tags can be embedded directly in the text.
• Multi‑speaker dialogue – Use voice tags or the dedicated /dev/dialogue endpoint to script conversations where each turn has its own voice.
• Custom voice‑pack generation – Via the Inno‑Clone‑Tuner (/dev/tune) you can tune a voice toward a short reference clip and optionally save the resulting .pt voicepack.
• Timestamped captions – Optional per‑word or per‑chunk timestamps for subtitle generation.
• Web UI – A built‑in browser UI (/web) for quick testing, voice‑mix preview, and long‑generation read‑along.
• Kubernetes / DigitalOcean deployment – Helm chart and deployment docs are provided.

How to run

1. Docker – Pull the appropriate image (e.g. ghcr.io/remsky/kokoro-fastapi-gpu:latest) and expose port 8880.
2. Docker‑Compose – Clone the repo, pick the docker/gpu (or cpu/rocm) folder and run docker compose up --build.
3. Direct – Install uv and system dependencies, then ./start-gpu.sh or ./start-cpu.sh.

All images ship with the model baked in; if you need to download it manually, a helper script is provided.

Typical usage (Python)

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8880/v1", api_key="not-needed")
resp = client.audio.speech.create(
    model="kokoro",
    voice="af_bella+af_sky",
    input="Hello world!",
    response_format="mp3",
)
resp.stream_to_file("out.mp3")

The same call works with requests if you prefer raw HTTP.

Why it matters

• Provides a self‑hosted, OpenAI‑compatible TTS backend, removing reliance on external paid APIs.
• The Kokoro‑82M model is small enough to run on consumer‑grade GPUs and even on CPU, yet supports many languages and voice‑mixing features that are rare in open‑source TTS stacks.
• The extensive API surface (streaming, voice‑tags, tuning, timestamps) makes it suitable for building voice assistants, audiobooks, interactive games, or any application that needs fast, controllable speech synthesis.

---

Links

• Model repo: https://huggingface.co/hexgrad/Kokoro-82M
• Demo Space: https://huggingface.co/spaces/Remsky/FastKoko
• Helm chart: docs/deployment/kubernetes.md
• Integration guides: Home Assistant, SillyTavern, OpenWebUI, etc.