instadeepai/Mava

What it solves

Mava is designed for researchers who need to experiment with multi-agent reinforcement learning (MARL) quickly. It addresses the need for high-performance, distributed implementations of MARL algorithms that can scale across hardware accelerators like GPUs and TPUs, while remaining simple enough for rapid iteration and modification.

How it works

Built on JAX, Mava uses single-file implementations to keep code clear and accessible. It supports two primary distribution architectures for scaling:

• Anakin: Used for environments written in JAX, allowing end-to-end JIT compilation of the entire training loop for maximum speed.
• Sebulba: Used for environments not written in JAX, enabling a hardware accelerator to interact with multiple CPU cores simultaneously.

It integrates with Hydra for configuration management and provides wrappers for various JAX-based MARL environment suites.

Who it’s for

Researchers and practitioners in the field of reinforcement learning who want to implement, test, and modify MARL algorithms at scale without the overhead of a complex modular library.

Highlights

• Diverse Algorithm Support: Includes implementations of PPO, Q-Learning, SAC, MAT, and Sable, supporting various learning paradigms like independent learners and centralized training with decentralized execution (CTDE).
• JAX-Powered Speed: Leverages pmap and vmap for blazingly fast experiments compared to non-JAX libraries.
• Statistically Robust Evaluation: Natively supports logging to JSON files following standard MARL-eval guidelines for easy plotting and aggregation.
• Flexible Environment Support: Out-of-the-box support for environments like Multi-Robot Warehouse, StarCraft Multi-Agent Challenge, and Multi-Agent Brax.