proroklab/VectorizedMultiAgentSimulator

VMAS is a vectorized differentiable simulator designed for efficient Multi-Agent Reinforcement Learning benchmarking. It is comprised of a vectorized 2D physics engine written in PyTorch and a set of challenging multi-robot scenarios. Additional scenarios can be implemented through a simple and modular interface.

What it solves

VMAS addresses the computational bottleneck in Multi-Agent Reinforcement Learning (MARL) benchmarking. Traditional physics engines are often too heavy to scale efficiently when simulating thousands of parallel environments with multiple interacting agents, which slows down the collection of training rollouts.

How it works

VMAS is a vectorized, fully-differentiable 2D physics engine built entirely in PyTorch. It uses tensor operations to simulate agents and landmarks, allowing it to run tens of thousands of parallel environments on accelerated hardware (GPUs). It employs holonomic motion models by default to focus on high-level multi-robot problem solving rather than low-level control, though it supports custom dynamics like differential drive, kinematic bicycle, and drone models.

Who it’s for

It is designed for researchers and developers in the MARL and robotics communities who need a fast, scalable environment to train and benchmark collective robot learning algorithms.

Highlights

  • High Scalability: Vectorization allows for massive parallelization across environments on GPUs.
  • Differentiable: Supports gradients flowing from output to input, enabling gradient-based optimization.
  • Broad Compatibility: Out-of-the-box wrappers for Gymnasium, RLlib, TorchRL, and OpenAI Gym.
  • Rich Physics Features: Supports elastic collisions, customizable gravity, joints, and various entity shapes (spheres, boxes, lines).
  • Integrated Sensors: Includes ray casting for distance-based sensors like LIDAR.
  • Modular Scenario Creation: Provides a framework to easily define custom multi-agent tasks and scenarios.

Related

  • Project
    google/brax

    A fast, fully differentiable physics engine written in JAX for large-scale rigid body simulation and reinforcement learning training.

  • Project
    facebookresearch/pytorch3d

    A PyTorch-based library for 3D Computer Vision research providing differentiable rendering and efficient tools for manipulating 3D meshes and point clouds.

  • Project
    kengz/SLM-Lab

    A modular PyTorch framework for Deep Reinforcement Learning that allows users to configure and run RL experiments via JSON specs across various Gymnasium environments.

  • Project
    EvolvingLMMs-Lab/lmms-engine

    A unified, high-performance training engine for multimodal models that integrates distributed training, kernel fusion, and advanced optimizers to enable scalable pretraining and fine-tuning.

  • Project
    facebookresearch/multimodal

    A PyTorch library for training state-of-the-art multimodal models at scale, providing modular building blocks and pretrained weights for models like CLIP and BLIP-2.