OpenAI Procgen and MineRL Competitions

OpenAI, in collaboration with AIcrowd, Carnegie Mellon University, and DeepMind, is co-organizing two competitions for NeurIPS 2020. These competitions utilize the Procgen Benchmark and MineRL to challenge the research community to improve sample efficiency and generalization in reinforcement learning (RL).

Procgen Competition: Improving Generalization and Sample Efficiency

The Procgen Competition aims to maximize agent performance using a fixed number of environment interactions across a diverse set of environments. This approach is designed to test an agent's ability to generalize to unseen situations and improve its sample efficiency.

Evaluation Metrics and Environment

Agents are evaluated across 16 publicly released Procgen Benchmark environments, as well as four secret test environments created specifically for the competition. By aggregating performance across these diverse settings, the organizers provide high-quality metrics to judge the underlying algorithms.

Design for Accessibility

Because Procgen environments are procedurally generated, they intrinsically require agents to generalize to new situations. The environments are designed to be fast and simple to use, allowing participants with limited computational resources to reproduce baseline results and iterate quickly on new methods.

MineRL Competition: Leveraging Human Demonstrations

The MineRL 2020 Competition focuses on developing algorithms that can efficiently leverage human demonstrations to solve complex, hierarchical, and sparse environments. This is a response to the fact that many state-of-the-art RL systems, such as AlphaStar and OpenAI Five, require an exponentially increasing amount of compute and simulator samples, making them difficult to apply to real-world problems where samples are expensive.

Technical Constraints and Goals

Participants must develop systems capable of obtaining a diamond in Minecraft from raw pixels. The competition imposes strict constraints to ensure the development of truly sample-efficient algorithms:

  • Sample Limit: Only 8,000,000 samples from the MineRL simulator.
  • Compute Limit: 4 days of training on a single GPU machine.
  • Data Source: Participants are provided the MineRL-v0 dataset, which contains over 60 million frames of human demonstrations.

Comparison to Standard RL

Following the MineRL 2019 Competition—where the top agent obtained an iron pickaxe—the 2020 challenge pushes the state-of-the-art further. While standard RL systems typically require hundreds of millions of environment interactions on large multi-GPU systems to achieve similar goals, the MineRL competition's constraints force a reliance on expert trajectories to minimize simulator interactions.

Validation and Anti-Overfitting Measures

To prevent hand-engineering of features or overfitting to the domain, the MineRL competition organizers train the top team's final round models from scratch using the same strict hardware, compute, and simulator-interaction constraints.

Sources