open-gigaai/giga-world-policy

What it solves

Robot policy learning often relies on World Action Models (WAMs) that predict both future visual scenes and robot actions. However, generating future videos during real-time operation creates massive computational overhead, making it too slow for closed-loop robot control.

How it works

GigaWorld-Policy-0.5 uses an action-centered approach. It uses future visual dynamics for dense supervision during training to help the model understand physical grounding, but switches to action-only decoding during inference to eliminate the need for video generation. To further increase speed, it employs a Mixture-of-Transformers architecture that separates visual dynamics and action generation into specialized experts, reducing the computation needed for action prediction. The project also utilizes an agent-based AutoResearch pipeline to automatically optimize training configurations and hyperparameters.

Who it’s for

Robotics researchers and developers building real-time control policies that require the physical grounding of world models without the latency of video generation.

Highlights

• Low Latency: Achieves 85ms inference latency on a local RTX 4090.
• Efficient Architecture: Uses a Mixture-of-Transformers to separate visual and action tasks.
• Action-Centered Inference: Removes the need to generate future videos at runtime.
• C++ Runtime: Provides a dedicated C++/CUDA runtime (wam.cpp) for real-time deployment using GGUF and ggml backends.