leggedrobotics/robotic_world_model
Repository for our papers: Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics and Uncertainty-Aware Robotic World Model Makes Offline Model-Based Reinforcement Learning Work on Real Robots
What it solves
This project provides a framework for model-based reinforcement learning in robotics, allowing robots to learn from "imagination" rather than relying solely on physical or simulated environment interactions. It addresses the need for robust policy optimization by enabling robots to learn dynamics models that predict future states, which can then be used to train policies either online (with simulator feedback) or offline (using static models).
How it works
The framework extends Isaac Lab to support two primary approaches: the Robotic World Model (RWM) and the Uncertainty-Aware Robotic World Model (RWM-U). Users train a neural network to act as a dynamics model—a simulator that predicts how the robot will respond to actions. Once trained, this model allows the robot to perform "imagination rollouts," where it practices tasks within the learned dynamics model. The system supports both online training, where the model and policy are updated concurrently with simulator data, and offline training, where the policy is optimized entirely within the static, learned model.
Who it’s for
This tool is designed for robotics researchers and developers working with model-based reinforcement learning who want to optimize policies for complex robots like the ANYmal D. It is particularly useful for those looking to bridge the gap between simulation-based training and real-world deployment by leveraging uncertainty-aware dynamics models.
Highlights
- Supports joint training of policies and neural dynamics models.
- Enables offline policy training using static dynamics models without needing a simulator.
- Provides tools for visualizing autoregressive imagination rollouts from learned dynamics.
- Includes specific configurations for the ANYmal D robot.
- Offers uncertainty estimation to improve the reliability of offline model-based reinforcement learning.
Related
- Project
- Project
- Project
- Project