AlayaLab/WildWorld

What it solves

WildWorld provides a large-scale, high-fidelity dataset designed to train AI models for dynamic world modeling. It specifically targets the creation of generative Action Role-Playing Games (ARPGs) by providing the necessary data to teach models how the world reacts to specific player actions in a photorealistic environment.

How it works

The project collects data from a AAA action role-playing game, automatically capturing a massive amount of information per frame. Each sample includes:

• Visuals: 720p RGB frames and lossless depth maps.
• State Data: Precise world coordinates, camera poses, and character stats (HP, stamina, attack power, and elemental resistances).
• Physicality: 3D joint positions (skeletons) for players, monsters, and weapons, including bone connectivity maps.
• Actions: Over 450 semantically meaningful actions, such as attacking or skill casting, paired with hierarchical natural language captions.

Who it’s for

Researchers and developers working on world models, generative video, embodied intelligence, and the development of AI-driven game environments.

Highlights

• Massive Scale: Over 108 million frames of data across diverse stages and monster species.
• Explicit State: Unlike simple video datasets, it provides internal game state (like health and animation IDs) for every frame.
• Long-Horizon Sequences: Includes gameplay clips spanning more than 30 minutes, allowing models to learn long-term dependencies.
• Comprehensive Annotations: Combines RGB, depth, 3D skeletons, and natural language descriptions of actions.