Kaixhin/imitation-learning

What it solves

This project provides a unified framework for implementing and benchmarking various deep imitation learning (IL) algorithms. It addresses the challenge of training agents to perform tasks by mimicking expert demonstrations, allowing researchers to compare different IL methods across standard robotic environments.

How it works

The library uses Soft Actor-Critic (SAC) as the base reinforcement learning algorithm and implements several imitation learning techniques, including:

• AdRIL, DRIL, GAIL, GMMIL, PWIL, and RED.
• Behavioral Cloning (BC) for pre-training.

It supports various configurations such as state-only imitation, absorbing state indicators, and mixing agent and expert data. It is benchmarked on Gym MuJoCo environments using D4RL expert data.

Who it’s for

This is primarily for researchers and developers in reinforcement learning and robotics who need a reliable, standardized way to test and evaluate different imitation learning algorithms.

Highlights

• Comprehensive Algorithm Suite: Includes a wide range of IL algorithms like GAIL and RED.
• Flexible Configuration: Offers extensive hyperparameters for discriminators, reward shaping (AIRL), and loss functions (BCE, Mixup, PUGAIL).
• Integrated Benchmarking: Built-in support for Gym MuJoCo and D4RL datasets.
• Hyperparameter Optimization: Supports Bayesian optimization via Ax and Hydra to find optimal settings for different trajectory counts.
• Parallel Execution: Ability to run training across multiple environments simultaneously.