TencentARC/GameHorizon

What it solves

GameHorizon addresses the lack of unified, large-scale data and evaluation frameworks for assessing how AI models handle gameplay across different time scales—ranging from immediate actions to long-term strategies in AAA games.

How it works

The suite consists of three integrated components:

• GameHorizon-Annotator: An automated pipeline that creates a "instruction pyramid" by categorizing gameplay into short-horizon operations, medium-horizon goals, and long-horizon strategies.
• GameHorizon-Data: A massive corpus featuring 5,000 hours of gameplay from 21 AAA titles, providing temporally aligned videos, actions, and multi-horizon instructions.
• GameHorizon-Bench: A dual-track evaluation system. The offline track uses thousands of multiple-choice questions (MCQs) for standardized testing, while the online track uses verifiable subtasks to test causal and thematic tasks in real-time environments.

Who it’s for

It is designed for researchers and developers building and evaluating AI models (such as multimodal LLMs) that need to understand, navigate, or automate complex gameplay in high-fidelity video game environments.

Highlights

• AAA Game Focus: Covers 21 different high-end game titles.
• Multi-Horizon Analysis: Evaluates models on three distinct levels of planning (operations, goals, and strategies).
• Extensive Dataset: Includes 5,000 hours of aligned gameplay data.
• Comprehensive Benchmarking: Features both offline MCQs and online task environments with failure localization via subtasks.