Ronchy2000/Multi-agent-RL

What it solves

This repository provides a structured learning path and implementation collection for Reinforcement Learning (RL) and Multi-Agent Reinforcement Learning (MARL). It specifically addresses the challenge of transitioning from basic single-agent RL algorithms to complex multi-agent scenarios, such as pursuit-evasion games (Predator-Prey) in continuous action spaces.

How it works

The project is organized as a progression of modules:

• Basic RL: Implementations of value iteration, policy iteration, Monte Carlo, Temporal Difference, DQN, and REINFORCE.
• Advanced Single-Agent RL: Implementations of Actor-Critic and DDPG.
• Multi-Agent RL: Extensions of single-agent algorithms to multi-agent environments using the PettingZoo MPE framework, featuring implementations of MADDPG, MATD3 (which uses twin Q-networks and delayed updates to reduce overestimation), and MAPPO/HAPPO (which support stochastic policies and heterogeneous agents with different observation dimensions).

Who it’s for

• Researchers and students studying Reinforcement Learning and Multi-Agent systems.
• Developers looking for concrete PyTorch implementations of MARL algorithms for continuous action spaces.
• Users interested in pursuit-evasion game simulations and swarm intelligence.

Highlights

• Comprehensive Path: Covers the journey from basic mathematical foundations to advanced MARL.
• Diverse MARL Algorithms: Includes MADDPG, MATD3, and HAPPO/MAPPO.
• Heterogeneous Support: HAPPO implementation allows for agents with different capabilities and observation dimensions.
• Optimized for Pursuit-Evasion: Specifically tuned for Predator-Prey tasks using PettingZoo.