Quantatirsk/qwen3-asr

Qwen3‑ASR — Local Speech‑Recognition API Service

What it is – A ready‑to‑run server that wraps the open‑source Qwen3‑ASR speech‑to‑text models (0.6 B and 1.7 B) and exposes them through familiar OpenAI‑compatible and Alibaba Cloud‑compatible HTTP and WebSocket endpoints. It can run on a GPU with the official vLLM backend or on a CPU/macOS with a vendored Rust backend, automatically picking the best runtime for the host.

Key capabilities

• Hybrid runtime – GPU → vLLM, CPU/macOS → Rust‑based QwenASR.
• Speaker diarization – Multi‑speaker detection (CAM++ model) with automatic speaker labeling.
• Real‑time streaming – WebSocket API (Paraformer + Qwen3‑ASR) for low‑latency transcription.
• Smart audio handling – VAD‑driven segmentation, far‑field noise filtering, and batch inference (2‑3× speedup on GPU).
• API compatibility – Works with the OpenAI /v1/audio/transcriptions endpoint and Alibaba Cloud Speech REST/WebSocket protocols, so existing client code can be pointed at the local server.
• Resource‑aware model selection – Chooses the 0.6 B or 1.7 B model based on VRAM; can be overridden with QWEN3_ASR_MODEL.

How to get it running

1. Docker (recommended) – Copy .env.example to .env, edit if you want an API key, then:

   docker-compose up -d               # GPU image (default)
   # or CPU only
   docker-compose -f docker-compose-cpu.yml up -d
   

   The service will be reachable at http://localhost:17003 with Swagger docs at /docs.
2. Custom GPU builds – The repo provides Dockerfile.gpu and build‑args to target CUDA 12.6, 12.8 (default) or 13.0.
3. Offline deployment – Run ./scripts/prepare-models.sh on a machine with internet, tar the resulting qwen3-asr-models-*.tar.gz, copy to the target host, extract, and start with Docker Compose.
4. Local development – Install Python 3.10+, then:

   uv sync                     # GPU stack (or ./scripts/sync_cpu_env.sh for CPU)
   source .venv/bin/activate
   python start.py
   

   macOS/Apple Silicon uses the Rust backend automatically.

Using the API

• OpenAI style (POST /v1/audio/transcriptions):

  from openai import OpenAI
  client = OpenAI(base_url="http://localhost:8000/v1", api_key="my_key")
  with open("audio.wav", "rb") as f:
      resp = client.audio.transcriptions.create(file=f, response_format="verbose_json")
  print(resp.text)
  

• Alibaba Cloud style (POST /stream/v1/asr or WebSocket /ws/v1/asr/qwen).
• Supported response formats: json, text, srt, vtt, verbose_json.
• Parameters let you toggle speaker diarization, request word‑level timestamps (available only for offline endpoints), and supply a language hint.

Supported models

Model ID	Size	Typical VRAM needed	Notes
qwen3-asr-1.7b	1.7 B	≥ 32 GB	Higher accuracy, multilingual (52 + dialects)
qwen3-asr-0.6b	0.6 B	< 32 GB	Lightweight, runs on CPU/macOS via Rust

License – MIT (see LICENSE).

Who might use this – Developers who need a self‑hosted, privacy‑preserving speech‑to‑text service that can be called with the same code they would use for OpenAI or Alibaba Cloud speech APIs, and who want optional speaker diarization and real‑time streaming without relying on external cloud providers.