NVlabs/GR00T-WholeBodyControl

What it solves

This project provides a unified framework for whole-body control (WBC) of humanoid robots, specifically targeting the Unitree G1. It solves the problem of creating separate, fragmented controllers for every specific motion (like walking, crawling, or jumping) by providing a generalist behavior foundation model that can generalize across a wide range of human-like movements.

How it works

The core of the project is SONIC, a humanoid behavior foundation model trained on large-scale human motion data (such as the Bones-SEED dataset). Instead of predefined motions, SONIC uses motion tracking as a scalable training task, allowing a single policy to produce natural movements.

The system supports several deployment modes:

• C++ Inference Stack: For high-performance deployment on real hardware using TensorRT.
• VR Teleoperation: Real-time human-to-robot motion transfer using PICO VR headsets for data collection and control.
• Kinematic Planner: A tool for real-time locomotion generation where users can steer the robot and adjust movement styles (e.g., "stealth" or "boxing") via keyboard or gamepad.
• Decoupled WBC: A hybrid approach using Reinforcement Learning (RL) for the lower body and Inverse Kinematics (IK) for the upper body.

Who it’s for

Robotics researchers and developers working with humanoid robots (specifically the Unitree G1) who need advanced motor skills, teleoperation capabilities, and data collection pipelines for training Vision-Language-Action (VLA) models.

Highlights

• Generalist Controller: A single model capable of walking, running, kneeling, jumping, and bimanual manipulation.
• Low-Latency Options: Specific model checkpoints optimized for responsive teleoperation and VLA execution.
• Comprehensive Tooling: Includes training scripts for PPO, data processing pipelines for motion capture data, and a C++ deployment stack.
• MotionBricks: A preview of a real-time latent generative model for interactive motion control in animation and robotics.