avaturn-live/avtr-1

AVTR‑1 – Live‑talking avatar model

What it is – AVTR‑1 is a flow‑matching‑based autoregressive neural network that takes a portrait image and two audio streams (speech + listening) and generates a 25 fps, lip‑synced video of a talking avatar. The repo ships the model weights, TensorRT‑accelerated inference code, and a demo‑ready backend that can be run locally or exposed as an API.

Key capabilities

• Live dialogue – renders speech and active‑listening gestures in real time (≈200 ms per 5‑frame chunk).
• Single‑GPU deployment – works on any Ampere‑or‑newer NVIDIA GPU with CUDA 12 and TensorRT 10.
• Multi‑speaker support – can generate both sides of a two‑person conversation by feeding each avatar its own speech track and the other side as the “listen” stream.
• Idle motion – produces subtle facial motion when no audio is provided.
• Production‑ready tooling – scripts to download weights, build GPU‑specific TensorRT engines, and run an interactive web demo.

How to try it

1. Install the lightweight package manager pixi (curl -fsSL https://pixi.sh/install.sh | sh).
2. Clone the repo and install dependencies:

   git clone https://github.com/avaturn-live/avtr-1.git
   cd avtr-1
   pixi install
   

3. Download the model weights (will prompt for a HuggingFace login):

   pixi run download
   

4. Build the TensorRT inference engines for your GPU:

   pixi run build‑trt‑engines   # or the per‑component commands
   

5. Run the interactive streaming demo in a browser:

   pixi run interactive-demo
   

6. For offline rendering you can use the provided CLI, e.g.:

   pixi run generate_offline --speech example/speaker_1.ogg --avatar elena --out elena.mp4
   

   (Swap the --speech/--listen arguments to render the other side of a dialogue.)

Performance snapshot (latency for a 5‑frame chunk, 25 fps output):

GPU	Latency	Real‑time factor
L40	84 ms	2.4×
A100	91 ms	2.2×
RTX 4060 Ti	166 ms	1.2×
RTX 3070	181 ms	1.1×
L4	202 ms	0.99×
RTX 3060 Ti	206 ms	0.97×
RTX 4060	232 ms	0.86×
A factor ≥ 1 means the GPU can keep up with the 25 fps stream.

Optional TURN server – The live demo uses WebRTC. If UDP cannot traverse your network, the README explains how to configure Cloudflare’s free Realtime TURN service (or any standard TURN server) via a small .env file.

Licensing

• Model & scripts – “AVTR‑1 Community License” (commercial use allowed for companies with < $10 M annual revenue; larger entities need a commercial agreement).
• Renderer – PolyForm Non‑commercial License 1.0.0 (commercial use requires a separate commercial license).
• Streamer backend – PolyForm Non‑commercial License 1.0.0 with a required notice and patent reservation (commercial use also requires a separate license).
• Third‑party components – InsightFace’s face detector/landmark models are research‑only; commercial deployments must obtain a license from InsightFace or replace them with permissively‑licensed alternatives.

Where to find more

• Project page & live demo: https://avaturn.live
• Model weights on HuggingFace: https://huggingface.co/avaturn-live/avtr-1
• Commercial inquiries: hello@avaturn.me

---

Bottom line – AVTR‑1 is a ready‑to‑run, GPU‑accelerated avatar talking‑head system that can be integrated into live‑streaming or video‑generation pipelines, with clear instructions for both interactive and offline use. The repository is well‑structured, provides performance numbers, and documents the licensing constraints that users must respect.