nv-tlabs/kimodo

Official implementation of Kimodo, a kinematic motion diffusion model for high-quality human(oid) motion generation.

What it solves

Kimodo is a kinematic motion diffusion model designed to generate high-quality 3D human and robot motions. It addresses the difficulty of creating precise, controllable 3D animations by allowing users to guide the generation process using natural language and specific kinematic constraints.

How it works

The model is trained on a large-scale optical motion capture dataset (up to 700 hours). It uses a diffusion-based architecture to generate motion sequences based on two primary inputs:

  1. Text Prompts: Natural language descriptions of the desired action.
  2. Kinematic Constraints: Precise controls including full-body pose keyframes, end-effector positions/rotations (hands and feet), and 2D paths or waypoints for the root movement.

It supports multiple skeleton types, including SOMA, Unitree G1, and SMPL-X, and provides a web-based interactive demo for timeline-based motion authoring.

Who it’s for

  • Robotics Researchers: Those needing to generate synthetic motion data for training physics-based policies (e.g., via ProtoMotions).
  • 3D Animators: Users looking for a fast way to author motions using text and keyframes.
  • AI Researchers: Developers working on motion generation and evaluation benchmarks.

Highlights

  • Multi-Skeleton Support: Compatible with SOMA, Unitree G1, and SMPL-X skeletons.
  • Extensive Controllability: Combines text-to-motion with precise kinematic constraints like end-effector control.
  • Integrated Benchmark: Includes a standardized evaluation pipeline and test suite to measure motion quality and text alignment.
  • Downstream Compatibility: Seamlessly integrates with MuJoCo for visualization and ProtoMotions for physics-based tracking.

Related

  • Project
    Tencent-Hunyuan/HY-Motion-1.0

    A series of billion-parameter text-to-3D human motion generation models that create skeleton-based animations from text prompts using Diffusion Transformers and Flow Matching.

  • Project
    NVlabs/ProtoMotions

    A GPU-accelerated simulation and learning framework for training physically simulated digital humans and humanoid robots, supporting large-scale motion learning and sim-to-real deployment.

  • Project
    aigc-apps/VideoX-Fun

    A video generation pipeline supporting CogVideoX-Fun and Wan 2.1 models for text, image, and video-to-video generation with advanced structural and camera controls.

  • Project
    adobe-research/custom-diffusion

    A fast and efficient method for fine-tuning text-to-image diffusion models to learn new concepts from a few images with minimal storage overhead.

  • Project
    jdh-algo/JoyVASA

    JoyVASA is a diffusion-based framework that animates human and animal portraits using audio, decoupling facial identity from motion to enable high-quality, long-form video generation.