Junchao-cs/SolarWM

Open data and scalable training for long-horizon video world models.

What it solves

SolarWM provides a unified, open-source framework for training and running interactive video world models. It addresses the complexity of working with heterogeneous video backbones by providing a single codebase that supports multiple state-of-the-art models (Wan2.2, LTX-2.5, and MiniMax-H3) while maintaining their native capabilities.

How it works

The project utilizes a three-stage training recipe to transform bidirectional video models into causal, interactive world models capable of long-horizon generation.

  1. Stage 0.5: Learns base video, text, and camera-conditioned representations using bidirectional flow matching.
  2. Stage 1: Uses teacher-forced autoregressive training to enable the model to learn denoising and finite-step flow maps simultaneously.
  3. Stage 2: Employs self-gradient forcing (SGF) to train the model on its own autoregressive rollouts, enabling stable, long-horizon interaction via sliding-window inference.

The framework is supported by a massive, open-source dataset of 1.43 million clips (25 TB) that includes frame-aligned metadata, camera geometry, and captions.

Who it’s for

This project is designed for researchers and developers working on video generation and world models who need a scalable, modular pipeline to train, fine-tune, or infer with large-scale video backbones.

Highlights

  • Unified Framework: Supports 5B–33B parameter models across different architectures (Wan2.2, LTX-2.5, MiniMax-H3) within one codebase.
  • Open Data: Provides a complete, 25 TB dataset with a flexible processing pipeline that decouples source data from training mixtures.
  • Long-Horizon Capability: Enables few-step interaction over arbitrarily long video sequences using sliding-window inference.
  • Advanced Rendering: Includes integration with SolarRenderer for structure-conditioned video generation.

Related

  • Project
    Ma-Zhuang/OmniNWM

    OmniNWM is a research codebase that builds a panoramic, multi‑modal world model for autonomous‑driving simulation. It jointly generates RGB, semantic, depth, and 3‑D occupancy maps from a vehicle trajectory, uses normalized Plücker ray‑maps for precise control, and provides occupancy‑based dense rewards for closed‑loop policy evaluation. The repo includes installation steps (including a manual patch to `transformers`), data preparation instructions for nuScenes, pretrained checkpoint download links, and scripts for inference, out‑of‑distribution testing, and staged training.

  • Project
    OpenMOSS/EasyWAM

    EasyWAM is a unified, high-performance research framework for training and evaluating World Action Models, designed to accelerate development through modular design and optimized training/inference pipelines.

  • Project
    haoyi-duan/WorldScore

    WorldScore is an open‑source benchmark for evaluating world‑generation models (3‑D, 4‑D, video). It provides a dataset, generation adapters, a suite of spatial‑temporal metrics, and a public leaderboard. Users register their model via a small YAML config and a Python class, generate videos, run the evaluation script, and submit the resulting JSON score.

  • Dispatch
    SANA-WM: Breaking the Minute Barrier in Open-Source World Modeling

    NVIDIA researchers introduce SANA-WM, a 2.6B parameter model capable of generating high-fidelity, 720p, minute-long videos with precise camera control on a single GPU.

  • Project
    OpenDriveLab/WorldEngine

    WorldEngine is an open-source post-training framework for autonomous driving that identifies rare, safety-critical driving scenarios from real-world logs and uses photorealistic simulation to train models to handle them safely.