OpenWAM-Official/OpenWAM

What it solves

OpenWAM addresses the challenge of building World-Action Models (WAMs) where design choices like model architecture, representation, and training strategy are often tightly coupled and hard to compare. It provides a modular research stack that turns these coupled decisions into interchangeable components, so researchers can systematically isolate, test, and improve each part. It also offers an open pretrained model (OpenWAM-alpha) trained on over half a billion frames of egocentric human and robot data, saving others from having to train from scratch.

How it works

OpenWAM is organized into three main parts:

• OpenWAM-Infra: A modular infrastructure for composing and comparing model, representation, training, inference, deployment, and evaluation choices. Users select from a menu of architectures (single, dual, or three-system) and backbones (video, visual, and language models) via Hydra YAML configs.
• OpenWAM-Study: Controlled experiments that derive practical principles for inheriting world knowledge, coupling world and action learning, and scaling across domains.
• OpenWAM-alpha: An open pretrained WAM that applies those principles, trained on 518.5M frames (about 6,400 hours) of egocentric human and robot data.

The repo includes scripts to download assets (video backbones, benchmark data, VLM backbones, etc.), train or fine-tune policies, and deploy them as a WebSocket server for inference. It supports many benchmarks (RoboTwin, LIBERO, LIBERO-plus, RoboCasa365, VLABench, etc.) and integrates with XPolicyLab.

Who it's for

Researchers and developers working on world-action models, robotics policies, or video-based AI agents who want a flexible, extensible platform for pretraining, fine-tuning, and evaluating their models. It is also useful for those who want to use a pretrained foundation model and fine-tune it for specific tasks, or who want to contribute to an open research effort in this area.

Highlights

• Modular Infrastructure: Swap model, representation, training, and deployment components easily via config files.
• Controlled Studies: Built-in experiments to derive design principles for world-action learning.
• Open Pretrained Model: OpenWAM-alpha trained on 518.5M frames of human and robot data.
• Multiple Architectures: Supports single, dual, and three-system designs with variants like joint self-attention or cross-attention.
• Multiple Backbones: Works with Wan2.2, Cosmos, Qwen3-VL, DINOv3, and other video/visual/VLM backbones.
• Wide Benchmark Support: Includes 8+ benchmarks for evaluation, plus integration with XPolicyLab.
• Released Datasets: Cleaned robot datasets (AgiBotWorld-Beta, RoboCOIN, InternData-A1, DROID) are available on HuggingFace.
• Open Source: Apache 2.0 license, with a project page, paper, and community channels.