ml-agents: a toolkit for training intelligent agents within Unity games and simulations

What it solves

It allows game developers and AI researchers to use Unity games and simulations as environments for training intelligent agents. This removes the need to build custom training environments from scratch, enabling the creation of complex NPC behaviors, automated game testing, and the evaluation of game design decisions.

How it works

The toolkit provides a Python API and a Unity SDK that connect a Unity scene to PyTorch-based machine learning algorithms. It supports various training methods, including Deep Reinforcement Learning (PPO, SAC, MA-POCA, self-play), Imitation Learning (BC, GAIL), and neuroevolution. Users can define curriculum learning scenarios and use environment randomization to create robust agents, then deploy them using a native cross-platform Inference Engine.

Who it’s for

• Game Developers: To create intelligent NPCs or automate the testing of game builds.
• AI Researchers: To use Unity's rich 3D/2D/VR/AR environments as a platform for evaluating new AI algorithms.
• Hobbyists: To easily implement state-of-the-art AI in their own games.

Highlights

• Diverse Training Methods: Supports reinforcement learning, imitation learning, and neuroevolution.
• Multi-Agent Support: Capable of training single-agent, cooperative multi-agent, and competitive multi-agent scenarios.
• Integration Ready: Can be wrapped as a Gym or PettingZoo environment for compatibility with other AI tools.
• Extensible: Allows users to add custom training algorithms and components.
• Scalable: Supports training across multiple concurrent Unity environment instances.