KlingAIResearch/LivePortrait

What it solves

LivePortrait is designed to create efficient, high-quality portrait animations. It allows users to animate a static portrait image (of humans, cats, or dogs) using a driving video or image, ensuring the resulting animation is realistic and controllable.

How it works

The system uses a portrait-warping renderer and keypoint detection. It supports two primary modes: Humans mode and Animals mode (which utilizes the X-Pose framework for keypoint detection). Users can provide a source image or video and a driving video to transfer motion. The project also supports "motion templates" (.pkl files) to speed up inference and protect privacy by removing the need for a driving video during the final render.

Who it’s for

This tool is for video creators, startups, and developers building applications for social media platforms (like Kuaishou, Douyin, and WeChat Channels) or those interested in AI-driven facial animation and portrait editing.

Highlights

• Multi-species support: Works for humans, cats, and dogs.
• Cros-platform compatibility: Supports Linux, Windows, and macOS (Apple Silicon).
• Cros-modal driving: Supports both video-to-video (v2v) and image-driven animation.
• Precise control: Includes regional control and precise portrait editing via a Gradio interface.
• Efficiency: Offers a torch.compile option for 20-30% faster inference on supported systems.