Winn1y/Awesome-Human-Motion-Video-Generation

What it solves

Researchers and developers working on human motion video generation — such as talking heads, portrait animation, dance videos, and text- or audio-driven avatars — often struggle to keep track of the fast-moving literature. This project collects and organizes hundreds of related papers (724+ papers and 65+ datasets are mentioned) into a single, searchable repository. It also provides a structured survey that breaks the field into clear stages, making it easier to understand how different methods relate and where open problems remain.

How it works

The repository groups research into three main driving conditions: Vision-driven, Text-driven, and Audio-driven, and it also includes LLM-based planning papers. The core contribution is a five-stage framework that describes the entire pipeline of digital-human video generation:

1. Input Phase — defining the driving source (vision, text, or audio) and the driving region (part, usually the face, or the whole body).
2. Motion Planning Phase — most works learn motion mappings through feature mapping, while a few use large language models for planning.
3. Motion Video Generation Phase — most methods are based on diffusion models, a smaller number on transformers.
4. Video Refinement Phase — optimizing specific parts like the face, lips, teeth, and hands.
5. Acceleration Phase — speeding up training and inference, aiming for real-time output.

The README also links to a full survey PDF (accepted by TPAMI), provides a timeline of updates, and lists recent paper additions with links to arXiv. The authors welcome contributions via pull requests to add missing work.

Who it’s for

This is primarily for researchers, graduate students, and engineers working on human-centric video generation — especially those focusing on 2D avatar animation, talking head synthesis, dance generation, and related tasks. It is also useful for newcomers who want a structured overview of the field and for anyone tracking recent developments through the frequent updates.

Highlights

• Comprehensive survey covering portrait animation, dance video generation, text-to-face, text-to-motion-video, and talking head topics.
• Five-stage decomposition of the generation pipeline — the README claims this is the first survey to offer such a framework.
• Organizes papers by driving condition (vision, text, audio) and distinguishes between motion planning and motion generation perspectives.
• Lists established baselines and evaluation metrics, with discussion of key challenges.
• Provides a set of potential future research directions.
• Regularly updated with new papers; includes links to arXiv and follows a versioned update history.
• Welcomes community contributions via pull requests and provides direct email contact for questions or missing works.