google-deepmind/mctx

Monte Carlo tree search in JAX

What it solves

Combining search algorithms with deep neural networks often requires implementations in fast compiled languages like C++, which can be difficult for researchers to use and modify. Mctx provides a high-performance, JAX-native implementation of Monte Carlo tree search (MCTS) that is accessible in Python, bridging the gap between execution speed and researcher usability.

How it works

Mctx implements MCTS algorithms (including AlphaZero, MuZero, and Gumbel MuZero) using JAX. It leverages JIT-compilation and operates on batches of inputs in parallel to maximize the efficiency of hardware accelerators. Users provide learned components—such as a representation function for the root state and a recurrent function for environment dynamics—which the library uses to construct search trees and propose actions.

Who it’s for

It is designed for AI researchers investigating search-based reinforcement learning agents who want the performance of compiled code without leaving the Python ecosystem.

Highlights

  • JAX-native: Fully supports JIT-compilation for significant computation speedups.
  • Parallelized Search: Operates on batches of inputs in parallel to optimize accelerator usage.
  • Algorithm Support: Includes implementations of AlphaZero, MuZero, and Gumbel MuZero.
  • Configurable: Designed to be heavily configurable to allow for the exploration of new search-based ideas.

Related

  • Project
    opendilab/LightZero

    A lightweight PyTorch toolkit combining Monte Carlo Tree Search and Deep Reinforcement Learning to standardize and accelerate research in MCTS-based decision-making algorithms.

  • Project
    haroldsultan/MCTS

    A Python implementation of Monte Carlo Tree Search (MCTS) used to experiment with decision-making in a state-based game environment.

  • Project
    patrick-kidger/equinox

    A JAX library that provides PyTorch-like syntax for building neural networks and models by treating them as PyTrees, ensuring full compatibility with the JAX ecosystem.

  • Project
    RobertTLange/evosax

    A high-performance JAX library implementing over 30 Evolution Strategies for efficient, vectorized neuroevolution on hardware accelerators.

  • Project
    AI-Hypercomputer/maxtext

    A high-performance, scalable LLM library and reference implementation written in JAX for training and post-training models on Google Cloud TPUs and GPUs.