AgileRL/AgileRL

AgileRL – A Reinforcement‑Learning Library with Built‑in RLOps

What it is – AgileRL is an open‑source Python library that bundles many state‑of‑the‑art reinforcement‑learning (RL) algorithms (on‑policy, off‑policy, offline, multi‑agent, bandits, and LLM fine‑tuning) together with a set of tools for evolutionary hyper‑parameter optimisation (HPO) and distributed training. The authors call this combination RLOps – the analogue of MLOps for RL workloads.

Key ideas

• Evolutionary HPO – instead of running dozens of separate experiments, AgileRL evolves a population of agents, mutating hyper‑parameters and network architectures every few thousand steps. The README claims up to 10× faster HPO than traditional Optuna‑based pipelines.
• Modular trainers – LocalTrainer (single‑machine) and an SDK/CLI for the cloud platform Arena let you train a single agent, a population, or a custom pipeline.
• Broad algorithm coverage – PPO, DQN (including Rainbow), DDPG/TD3, CQL, ILQL, MADDPG, MATD3, IPPO, NeuralUCB/NeuralTS, plus a special LLM fine‑tuning flow (CISPO) for long‑horizon, multi‑turn tasks.
• Distributed / multi‑agent support – works with Gymnasium vectorised environments and PettingZoo parallel APIs; can scale across many GPUs via the Arena RLOps platform.

How you use it

# Train a single DQN agent on LunarLander
from agilerl import LocalTrainer
trainer = LocalTrainer(algorithm="DQN", environment="LunarLander-v3")
population, fitnesses = trainer.train()

For evolutionary HPO you simply enable hpo=True and optionally set pop_size:

trainer = LocalTrainer(
    algorithm="DQN",
    environment="LunarLander-v3",
    training=TrainingSpec(pop_size=4),
    hpo=True,
)
population, fitnesses = trainer.train()

You can also drive training from a YAML manifest or the command‑line (python -m agilerl.train path/to/config.yaml).

Arena – the managed RLOps service

• A separate agilerl‑arena package provides a Python SDK (ArenaClient) and a CLI (arena …).
• With these you can upload custom environments, submit training jobs, and deploy trained agents on cloud GPUs without managing the infrastructure yourself.

Installation

pip install agilerl               # core library
pip install agilerl[box2d]        # optional Box2D physics for classic control
pip install agilerl[llm]          # LLM fine‑tuning extras
pip install agilerl-arena         # Arena SDK/CLI (also a base dependency of agilerl)

Development mode is supported via pip install -e . or pip install git+https://github.com/AgileRL/AgileRL.git@main.

Documentation & community

• Full docs at https://docs.agilerl.com (tutorials for single‑agent, multi‑agent, bandits, curriculum learning, custom networks, and LLM fine‑tuning).
• A Discord server and an online benchmark page showcase the claimed 10× speed‑up in HPO and state‑of‑the‑art LLM fine‑tuning results.

Who it’s for Researchers or engineers who need a ready‑made RL toolbox that reduces the plumbing around hyper‑parameter search and scaling, especially those interested in trying evolutionary HPO or training LLMs with RL.

---

Bottom line – AgileRL is a genuine, actively maintained RL framework that couples a wide algorithm suite with evolutionary hyper‑parameter optimisation and a cloud‑native RLOps platform (Arena). It aims to make RL experimentation faster and more reproducible.