opendilab/LightZero

[NeurIPS 2023 Spotlight] LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarios (awesome MCTS)

What it solves

LightZero provides a lightweight and efficient toolkit for implementing and researching Monte Carlo Tree Search (MCTS) combined with Deep Reinforcement Learning (RL). It aims to standardize the MCTS+RL algorithm family to accelerate research and application development across various decision-making problems.

How it works

LightZero is built on PyTorch and uses a modular framework consisting of three core components:

  • Model: Defines the network structure and forward propagation.
  • Policy: Manages how the network is updated and interacts with the environment through learning, collecting, and evaluation processes.
  • MCTS: Handles the search tree structure and its interaction with the Policy, with implementations in both Python and C++ to optimize the most time-consuming parts of the algorithm.

Who it’s for

It is designed for researchers and developers working on decision-making AI, specifically those using MCTS and RL to solve games (like Go or Atari) or complex scientific and control problems.

Highlights

  • Algorithm Variety: Integrates multiple MCTS-based algorithms including AlphaZero, MuZero, EfficientZero, UniZero, and ReZero.
  • Efficient Computing: Employs mixed heterogeneous computing to improve performance.
  • Broad Environment Support: Compatible with a wide range of environments including Atari, MuJoCo, MiniGrid, and various board games.
  • Developer Friendly: Includes detailed documentation, function call graphs, and network structure diagrams to simplify code navigation.

Related

  • Project
    google-deepmind/mctx

    A JAX-native library providing high-performance implementations of Monte Carlo tree search algorithms like AlphaZero and MuZero for reinforcement learning.

  • Project
    haroldsultan/MCTS

    A Python implementation of Monte Carlo Tree Search (MCTS) used to experiment with decision-making in a state-based game environment.

  • Project
    yihaosun1124/OfflineRL-Kit

    A PyTorch library for offline reinforcement learning that provides a standardized framework and implementations of state-of-the-art model-free and model-based algorithms.

  • Project
    alibaba/TorchEasyRec

    A PyTorch-based recommendation system framework that provides a library of production-ready deep learning models for matching, ranking, and generative recommendation.

  • Project
    pykt-team/pykt-toolkit

    A PyTorch-based library for training and benchmarking deep learning knowledge tracing models with standardized data preprocessing and multiple model implementations.