AlayaLab/AlayaWorld

What it solves

AlayaWorld addresses the challenge of generating long-horizon, consistent videos that are interactive and playable. It solves the problem of "drift" and visual artifacts that typically accumulate during long video rollouts, ensuring that revisited locations remain recognizable and the environment stays stable over minute-long sequences.

How it works

AlayaWorld is an autoregressive world model that uses a combination of spatial and temporal memory to maintain consistency. It utilizes a 3D cache for spatial recall (reprojected to the current view) and a compressed frame-history embedding for temporal continuity. To prevent error compounding, it employs an "error bank" that re-injects artifacts back into the training process. For real-time performance, it uses few-step DMD distillation to reduce the number of steps required for generation, allowing for live camera control and prompt switching at chunk boundaries.

Who it’s for

This project is designed for researchers and developers working on video generation, world models, and interactive AI environments.

Highlights

• Interactive Control: Supports real-time camera navigation via keyboard and mid-generation prompt switching.
• Long-Horizon Stability: Maintains visual consistency over minute-long videos using specialized memory systems and error-injection training.
• Real-time Performance: Achieves low-latency interaction through DMD distillation and short temporal chunks.
• Comprehensive Pipeline: Provides full-stack training and inference code, including multiple stages of pretraining and SFT.