simchowitzlabpublic/nano-world-model
A Minimalist, Batteries-included Repository for Advancing World Model Science.
What it solves
Nano World Model provides a minimalist, open-source framework for training video world models. It addresses the complexity of future video prediction by implementing a streamlined pipeline for training, validating, and evaluating models that can predict subsequent frames in a video sequence based on previous states and actions.
How it works
The project is based on diffusion-forcing, a technique used to generate future video frames. It uses a unified pipeline managed by a Hydra-based configuration system to handle training and inference. The model can perform autoregressive long-rollouts, where it predicts frames sequentially, and can integrate action injection to influence the predicted outcomes. It supports various datasets including DINO-WM, RT-1, and CSGO.
Who it’s for
This tool is designed for researchers and developers working on video prediction, world models for robotics, and embodied intelligence who need a clean, transparent codebase for conducting ablations and testing new architectures.
Highlights
- Minimalist Design: Inspired by NanoGPT, it focuses on ease of use and with minimal dependencies for fast setup.
- Diverse Applications: Supports long-horizon video generation, MPC-style planning (using CEM), and 3D point cloud reconstruction from rollout videos.
- Scientific Transparency: Provides full open-source access to model checkpoints and head-to-head ablation studies on model scale and action injection.
- Broad Dataset Support: Compatible with multiple environments including point mazes, robotic manipulation (PushT), and game environments (CSGO).