NVlabs/GR00T-WholeBodyControl

What it solves

This project provides a framework for advanced whole-body control (WBC) of humanoid robots, specifically targeting the Unitree G1. It solves the problem of creating natural, versatile movements (like walking, crawling, and jumping) without needing to build separate controllers for every single predefined motion, while also enabling high-fidelity teleoperation and data collection for further AI training.

How it works

The core of the project is SONIC, a humanoid behavior foundation model trained on large-scale human motion data (such as the Bones-SEED dataset). Instead of rigid programming, SONIC uses motion tracking as a training task, allowing a single unified policy to produce a wide range of behaviors. The system includes:

• Model Checkpoints: Various versions of the SONIC controller, including low-latency versions for responsive teleoperation and v1.1 for heading-stable control.
• C++ Inference Stack: A high-performance deployment layer using TensorRT for real-time execution on physical hardware.
• VR Teleoperation: A system using PICO VR headsets to transfer human movements to the robot in real-time.
• Kinematic Planner: A tool for real-time locomotion generation via keyboard or gamepad.
• MotionBricks: A latent generative model for interactive motion control in animation and robotics.

Who it’s for

• Robotics Researchers: Those developing humanoid behavior models and VLA (Vision-Language-Action) workflows.
• Robot Operators: Users needing high-precision whole-body teleoperation for data collection or interactive control.
• AI Engineers: Developers training or fine-tuning humanoid motor skills using reinforcement learning (PPO) and large motion datasets.

Highlights

• Foundation Model Approach: Uses a single policy for diverse behaviors (walking, running, kneeling, boxing) rather than separate controllers.
• VR-to-Robot Transfer: Supports real-time 3-point teleoperation with wrist-pose tracking.
• End-to-End VLA Workflow: Provides a pipeline to collect teleop data, fine-tune models, and deploy them.
• High-Performance Deployment: Includes a C++ stack for low-latency inference on real hardware.
• Large-Scale Data Support: Compatible with the Bones-SEED dataset containing over 142K human motions.