Farama-Foundation/MO-Gymnasium

Multi-objective Gymnasium environments for reinforcement learning

What it solves

It provides a standardized way to develop and compare multi-objective reinforcement learning (MORL) algorithms. Previously, researchers needed a consistent API and a set of benchmark environments that could handle multiple reward signals simultaneously rather than a single scalar value.

How it works

MO-Gymnasium extends the standard Gymnasium API. While it maintains the same basic structure for interacting with environments, it modifies the reward system: instead of returning a single number, the step function returns a vectorized reward as a numpy array. It also includes a LinearReward wrapper that allows users to optionally convert these multiple rewards back into a single scalar value using weights.

Who it’s for

Researchers and developers working on multi-objective reinforcement learning who need a reliable toolkit for benchmarking and testing their algorithms against standard environments.

Highlights

  • Standardized API for multi-objective reinforcement learning.
  • Includes a variety of environments from MORL literature and multi-objective versions of classical environments like MuJoco.
  • Strict versioning of environments to ensure research reproducibility.
  • Compatible with Gymnasium API patterns.

Related

  • Project
    Farama-Foundation/Gymnasium

    A standard Python API and environment suite for developing and comparing reinforcement learning algorithms.

  • Project
    PKU-Alignment/safety-gymnasium

    A scalable and customizable Safe Reinforcement Learning benchmark library providing a standardized set of environments for evaluating safety-constrained AI agents.

  • Project
    Farama-Foundation/PettingZoo

    A Python library for multi-agent reinforcement learning research that provides a standardized API and a diverse set of environments, similar to a multi-agent version of Gymnasium.

  • Project
    Farama-Foundation/Gymnasium-Robotics

    A collection of Reinforcement Learning robotic environments powered by the MuJoCo physics engine and the Gymnasium API for training AI agents in manipulation and navigation tasks.

  • Project
    LucasAlegre/morl-baselines

    MORL‑Baselines is a PyTorch library of reliable multi‑objective reinforcement‑learning algorithms (single‑ and multi‑policy, SER/ESR). It follows the MO‑Gymnasium API, provides utilities, automatic W&B logging, and integrates with the Open RL Benchmark for reproducible experiments.