nicklashansen/tdmpc2

Code for "TD-MPC2: Scalable, Robust World Models for Continuous Control"

What it solves

TD-MPC2 is designed to create robust, scalable agents for continuous control tasks. It addresses the challenge of training agents that can perform well across a wide variety of environments (such as robotics and physics simulations) using a single set of hyperparameters, reducing the need for tedious per-task tuning.

How it works

TD-MPC2 is a model-based reinforcement learning algorithm that uses world models to predict future states and rewards. It supports both single-task online reinforcement learning (learning from experience) and multi-task offline reinforcement learning (learning from pre-collected datasets). The system can process both state-based and pixel-based (RGB) observations and can scale up to models with 317 million parameters to handle dozens of tasks across different domains and embodiments.

Who it’s for

This project is for researchers and developers working in robotics, embodied intelligence, and reinforcement learning, specifically those focusing on continuous control and multi-task learning.

Highlights

  • Broad Compatibility: Supports 104 continuous control tasks across DMControl, Meta-World, ManiSkill2, and MyoSuite.
  • Scalability: Capable of training a single agent (up to 317M parameters) to perform 80 different tasks.
  • Versatility: Works with both state and pixel observations.
  • Robustness: Performs competitively against model-free and model-based methods using a single set of hyperparameters.

Related

  • Project
    Farama-Foundation/Metaworld

    An open source benchmark for developing and evaluating multi-task and meta-reinforcement learning algorithms for continuous control robotic manipulation.

  • Project
    danijar/dreamerv3

    A scalable reinforcement learning algorithm that uses world models to master diverse control tasks with a fixed set of hyperparameters.

  • Project
    AgentR1/Agent-R1

    A modular framework for Agentic Reinforcement Learning that trains multi-step LLM agents by treating each interaction turn as a step-level MDP transition.

  • Project
    Farama-Foundation/MPE2

    A set of communication-oriented multi-agent environments for training AI agents to coordinate, communicate, and compete in particle-based simulations.

  • Project
    FilippoAiraldi/mpc-reinforcement-learning

    A library for training model-based Reinforcement Learning agents using Model Predictive Control as a function approximation to learn optimal control policies.