amap-cvlab/ABot-World

What it solves

ABot-World addresses the limitation of passive video playback in AI-generated worlds, allowing users to interact with and control the environment in real time. It specifically solves the problem of "scene lock-in" and fixed video lengths, enabling infinite, open-ended world generation that can run on consumer-grade hardware.

How it works

The system uses a causal student model (ABot-World-0-5B-LF) trained with a technique called LongForcing. This allows the model to imagine new scenes and dynamics during the rollout process without needing to switch prompts. It is optimized for high-performance desktop inference, utilizing specialized kernels (like SageAttention and LightX2V) to achieve real-time interaction speeds on a single GPU.

Who it’s for

This project is designed for researchers and developers working on world simulators, interactive AI environments, and real-time video generation, as well as users with high-end consumer GPUs (specifically tested on the NVIDIA RTX 5090).

Highlights

• Real-Time Performance: Achieves 720p resolution at 16 FPS with 1.2s latency on a single RTX 5090.
• Action-Driven Control: Enables continuous exploration of the world based on user actions rather than static playback.
• Infinite Rollout: Supports open-ended generation that exceeds traditional fixed video-length limits.
• Efficient Memory Usage: Operates within 19GB of GPU memory.