MinghaoFu/lewam

What it solves

LeWAM provides a framework for end-to-end world and action modeling, specifically designed to help autonomous agents understand their environment and plan actions to reach specific goals or complete tasks.

How it works

It utilizes Joint-Embedding Predictive Architectures (JEPAs) to model the world. The system supports two primary modes: goal-conditioned visual planning, where the model uses gradient-based optimization to plan sequences of actions to reach a target state, and task completion, which uses behavior cloning to perform specific tasks.

Who it’s for

This project is for researchers and developers working on robotics and autonomous agents who need a robust framework for training and evaluating world models on simulation data.

Highlights

• Supports both gradient-based visual planning and direct behavior cloning.
• Includes pre-trained checkpoints and simulation datasets for various tasks like Push-T, PointMaze, and ToolHang.
• Built on modular configuration files for easy training and evaluation customization.
• Compatible with standard simulation environments like MuJoCo.