Tencent-Hunyuan/HY-WorldPlay

What it solves

HY-World 1.5 (WorldPlay) addresses the trade-off between generation speed and memory in interactive world modeling. It solves the problem of creating immersive 3D worlds that can be interacted with in real-time while maintaining long-term geometric consistency, avoiding the "memory attenuation" or drift often seen in long-horizon video generation.

How it works

WorldPlay is a streaming video diffusion model that uses several key architectural designs to achieve its goals:

• Dual Action Representation: Enables robust control of the world via keyboard and mouse inputs.
• Reconstituted Context Memory: Dynamically rebuilds context from past frames and uses temporal reframing to keep geometrically important frames accessible.
• WorldCompass: A Reinforcement Learning (RL) post-training framework that improves action-following and visual quality.
• Context Forcing: A distillation method that aligns memory context between teacher and student models to enable real-time speeds without error drift.

The system generates video in chunks of 16 frames, predicting the next chunk based on user actions and reconstituted memory.

Who it’s for

This project is for researchers and developers working on 3D world modeling, interactive AI environments, and real-time video generation, as well as those interested in first-person and third-person perspective simulation in both real and stylized environments.

Highlights

• Real-time Performance: Generates long-horizon streaming video at 24 FPS.
• Comprehensive Framework: Open-sources the entire pipeline, including data, training, RL post-training, and inference deployment.
• Flexible Control: Supports intuitive camera trajectory control via pose strings (e.g., w-31 for forward movement).
• Multiple Model Options: Provides bidirectional, autoregressive, and distilled models for different speed and quality trade-offs.