kleinlee/DH_live

What it solves

DH_live provides a high-performance, ultra-lightweight solution for creating 2D digital humans that can run in real-time on mobile browsers and low-power devices without requiring a GPU or extensive training.

How it works

The project uses a specialized architecture consisting of a video generation model (DINet_mini), a speech feature model (LSTM), and face detection/mesh models to animate a digital human based on audio input. It processes video data into a lightweight format that can be streamed to a web client, where inference happens directly on the device. Recent updates have replaced dependencies like MediaPipe with more efficient matting algorithms (matanyone2) for background removal.

Who it’s for

Developers and businesses looking to integrate real-time, interactive digital avatars into websites, mobile apps, or mini-programs without the overhead of heavy server-side GPU rendering.

Highlights

• Ultra-low compute requirements: Inference for a single frame requires as little as 39 Mflops, making it faster than most mobile face detection algorithms.
• No training needed: The system is open-box and ready to use without complex training processes.
• Extremely small footprint: Web resources can be compressed to under 3MB.
• Cross-platform support: Runs on Windows, Linux, macOS, and mobile browsers (iOS/Android).
• Full pipeline integration: Includes a complete real-time dialogue flow encompassing VAD, ASR, LLM, TTS, and the digital human animation.