AsrServe – Self‑hosted Speech‑to‑Text Service

What it is

• A ready‑to‑run API that turns spoken audio into written text. It works locally, so no cloud calls are needed, and it supports both real‑time streaming and batch file transcription.
• The service bundles the latest open‑source speech models (Confucius4‑R2T2 for recognition, CT‑Transformer for punctuation, Nemotron‑3 for speaker diarisation) and selects the most efficient inference path for the hardware you have (CUDA GPU, CPU, or Apple Silicon).

Key features

• State‑of‑the‑art accuracy – uses the Confucius4‑R2T2 model and a punctuation transformer that restores sentence boundaries after the whole file is processed.
• Realtime & offline modes – the same model weights power a low‑latency WebSocket stream and a batch /v1/audio/transcriptions endpoint.
• Speaker diarisation – identifies up to 8 speakers in a recording using the Nemotron‑3 diarisation model.
• Word‑level timestamps – optional timestamps from the Qwen3‑ForcedAligner.
• OpenAI‑compatible API – endpoints follow the OpenAI /v1/audio/transcriptions and /v1/realtime conventions, so existing SDKs work out of the box.
• Hot‑words & prompts – you can supply a list of terms and a recording context (up to 2048 characters) that guide the recogniser without forcing replacements.
• Cross‑platform – Docker images for GPU (quantatrisk/asrserve:gpu), CPU (:cpu), and Ascend 910B (:ascend). Native macOS support via a bundled Rust backend, no Docker needed.
• Easy deployment – Docker‑Compose files for GPU/CPU, a simple uv‑based Python start script for macOS, and scripts to pre‑download models for offline machines.

How to get started

# GPU (Docker) – pull image and start the service
docker compose up -d               # uses quantatrisk/asrserve:gpu

# CPU only (Docker)
docker compose -f compose.cpu.yml up -d   # pulls quantatrisk/asrserve:cpu

# macOS native (no Docker)
uv sync --frozen && ./scripts/build-rust.sh
HF_HOME="$PWD/models/huggingface" uv run --no-sync python start.py   # http://localhost:17003

• The first run downloads the required model files into ./models.
• The service listens on port 17003; Swagger docs are at /docs.

Typical API calls

• File transcription (curl example in the README) – send a WAV file, optional hotwords, prompt, enable_speaker_diarization, and word_timestamps flags.
• Realtime streaming – open a WebSocket at ws://localhost:17003/v1/stream and stream 16 kHz PCM16 audio, or use the OpenAI‑compatible endpoint /v1/realtime?intent=transcription with Base64‑encoded 24 kHz audio.

When it shines

• Private or regulated environments where sending audio to external services is not allowed.
• Applications that need low‑latency transcription (voice assistants, live captioning) and also occasional batch processing of longer recordings.
• Multi‑speaker meetings where speaker labels are useful.

Limitations / considerations

• Requires a decent GPU for the fastest performance; CPU mode is slower but still functional.
• Punctuation restoration runs on CPU, so it adds a post‑processing step after the main transcription.
• The hot‑word / prompt length is capped at 2048 characters; longer inputs return HTTP 400.
• Model files are large (hundreds of MB to a few GB) and need to be downloaded on first start.

Where to learn more

• Live demo: https://asr.ieeio.com
• Full documentation: docs/ folder (deployment, realtime protocol, OpenAI realtime examples).
• Release notes detail accuracy improvements and breaking changes.

Contact

• Email: pengzhia@gmail.com
• WeChat QR code is included in the repo for direct messaging.