dexmal/opendw

An Open-Source World Model for Action-Conditioned Embodied Intelligence.

What it solves

DW05 is a multimodal world model designed for embodied intelligence. It helps robots understand how specific actions will lead to future outcomes by predicting future video sequences, generating actions, and estimating state-values based on current observations.

How it works

DW05 uses a Mixture-of-Experts (MoT) framework built on a Wan backbone. It processes inputs including language, images/videos, robot type, state, and actions. This backbone feeds into three specialized expert heads: one for video prediction, one for action generation, and one for state-value estimation.

Who it’s for

This project is intended for researchers and developers working on robot learning, embodied AI, and world models for robotic decision-making.

Highlights

  • Unified Framework: Combines video prediction, action generation, and value estimation in a single model.
  • Multimodal Inputs: Supports language, visual observations, and robot state/proprioception.
  • Pretrained Weights: Provides base and fine-tuned (SFT-RBT) checkpoints via Hugging Face.
  • Action-Conditioned Rollouts: Capable of predicting future video based on robot observations and actions.

Related

  • Project
    PetoiCamp/OpenCatEsp32-Quadruped-Robot

    OpenCatESP32 is the firmware that runs Petoi’s open‑source quadruped robots (Bittle X, Nybble Q, and the upcoming Quaddle) on an ESP32‑based BiBoard. It provides real‑time gait control, a motion‑recording “Puppet Mode”, and enough processing power to add vision or reinforcement‑learning experiments, all programmable via the Arduino IDE.

  • Project
    vuer-ai/vuer

    An event-driven, declarative visualization toolkit for interacting with dynamic 3D and robotics data across browsers, mobile, and VR/AR.

  • Project
    facebookresearch/momentum

    A library of foundational algorithms and numerical optimization solvers for human kinematic motion and body tracking.

  • Project
    lucidrains/q-transformer

    Q‑Transformer is a PyTorch library that implements the autoregressive Q‑function approach for scalable offline reinforcement learning, targeting robotic tasks that combine video observations and language instructions. It provides a vision‑transformer model, data‑collection agent, replay‑memory dataset, and Q‑learning trainer, all installable via `pip install q-transformer`.