ai4co/rl4co

What it solves

RL4CO provides a unified benchmark and framework for applying Reinforcement Learning (RL) to Combinatorial Optimization (CO) problems. It aims to decouple the scientific research of RL-based CO algorithms from the engineering overhead of implementing them, facilitating reproducible research.

How it works

Built on top of PyTorch, TorchRL, TensorDict, PyTorch Lightning, and Hydra, the framework provides modular components for solving optimization problems. It supports three main types of policies:

• Constructive policies: These build a solution from scratch. This includes Autoregressive (AR) policies that build solutions step-by-step, and Non-Autoregressive (NAR) policies that predict heuristics (like heatmaps) to guide construction.
• Improvement policies: These focus on refining and improving a pre-existing solution.

Who it’s for

Researchers and developers working on AI-driven combinatorial optimization, specifically those using reinforcement learning to solve complex routing or scheduling problems.

Highlights

• Unified Framework: Integrates multiple RL libraries to provide a standardized environment for CO benchmarks.
• Modular Design: Features reusable components like environment embeddings that can easily be swapped to solve new problems.
• Flexible Policies: Supports both constructive (AR and NAR) and improvement-based RL policies.
• High Performance: Leverages GPU-vectorized environments via TorchRL and high-performance training wrappers via PyTorch Lightning.