yihaosun1124/OfflineRL-Kit

An elegant PyTorch offline reinforcement learning library for researchers.

What it solves

OfflineRL-Kit is a PyTorch-based library designed to simplify the implementation and experimentation of offline reinforcement learning (RL). It provides a standardized framework for researchers to implement, test, and compare state-of-the-art offline RL algorithms without having to build the infrastructure from scratch.

How it works

The library provides a modular architecture where users can define actor and critic backbones (using MLPs), setup policies (such as CQL or IQL), and use a MFPolicyTrainer to handle the training loop. It integrates with the D4RL benchmark and supports both model-free and model-based offline RL approaches. To facilitate research, it includes a powerful logging system that outputs to CSV, TensorBoard, and JSON, and integrates with Ray for parallel hyperparameter tuning.

Who it’s for

It is primarily intended for AI researchers and developers working on offline reinforcement learning who need a scalable and elegant framework to implement new algorithms or benchmark existing ones.

Highlights

  • Comprehensive Algorithm Support: Includes model-free algorithms like CQL, TD3+BC, IQL, and EDAC, as well as model-based algorithms like MOPO, COMBO, and RAMBO.
  • Scalable Framework: Designed for high scalability, allowing users to build new algorithms with minimal code.
  • Integrated Tuning and Logging: Built-in support for parallel tuning via Ray and a structured logging system for managing experiments.
  • Pure PyTorch: Built entirely on PyTorch for flexibility and ease of use.

Related

  • Project
    VLARLKit/VLARLKit

    A PyTorch library for training Vision-Language-Action (VLA) models using reinforcement learning, featuring a decoupled architecture to prevent dependency conflicts between models and simulators.

  • Project
    meta-pytorch/torchforge

    A PyTorch-native agentic RL library that separates infrastructure concerns from model concerns to simplify scalable reinforcement learning experimentation.

  • Project
    pykt-team/pykt-toolkit

    A PyTorch-based library for training and benchmarking deep learning knowledge tracing models with standardized data preprocessing and multiple model implementations.

  • Project
    rlcode/reinforcement-learning

    A collection of easy-to-read, standalone PyTorch implementations of core reinforcement learning algorithms, from basic Grid World to deep RL for Atari games.

  • Project
    kengz/SLM-Lab

    A modular PyTorch framework for Deep Reinforcement Learning that allows users to configure and run RL experiments via JSON specs across various Gymnasium environments.