RLinf/RLinf

RLinf: Reinforcement Learning Infrastructure for Embodied and Agentic AI

What it solves

RLinf provides a scalable and flexible infrastructure for training reinforcement learning (RL) models, specifically targeting Embodied AI (robotics) and Agentic AI. It bridges the gap between simulation and the real world by offering a unified backbone for data collection, supervised fine-tuning (SFT), RL training, and deployment.

How it works

RLinf acts as a robust training backbone that supports a wide array of hardware (NVIDIA, AMD, Huawei Ascend, Moore Threads, Biren GPUs) and software ecosystems (PyTorch). It implements a unified robotics interface to manage hardware, tasks, and rollout wrappers. The system supports various training paradigms, including:

  • VLA (Vision-Language-Action) Training: Scalable RL training for VLA models across different architectures and simulators.
  • Real-World Integration: Treating physical robots as first-class resources for online policy learning.
  • System Optimizations: Using techniques like DynaRL for dynamic compute reallocation and FUSCO for accelerated MoE communication to increase training throughput.
  • World Model Integration: Supporting RL fine-tuning using world models as simulators.

Who it’s for

It is designed for AI researchers and engineers working on embodied intelligence, robotic control, and autonomous agents who need a high-performance framework to move models from simulation to real-world hardware.

Highlights

  • Broad Hardware Support: Compatible with multiple GPU vendors beyond NVIDIA, including ROCm, CANN, and MUSA.
  • Deep Robotics Integration: Official adoption by Isaac Lab v3.0.0 and support for various robots like Franka and XSquare Turtle2.
  • Diverse RL Support: Implements GRPO, SAC, IQL, and DAgger, as well as support for SFT and RL fine-tuning of VLA models.
  • Performance Optimized: Achieves significant speedups in simulators (e.g., 25× for BEHAVIOR) through system-level optimizations.

Related

  • Project
    areal-project/AReaL

    A large-scale asynchronous reinforcement learning infrastructure optimized for training high-performance reasoning and agentic AI models.

  • Project
    verl-project/verl

    A flexible and efficient RL training library for LLMs that simplifies the implementation of RLHF algorithms and scales to trillion-parameter models.

  • Project
    FluxVLA/FluxVLA

    FluxVLA Engine is an open‑source, end‑to‑end platform for training, evaluating, and deploying vision‑language‑action (VLA) models for robots. It offers unified configuration, modular model back‑ends, benchmark suites (LIBERO, RoboCasa, RoboDojo), and deployment tools (Docker, Jetson Orin, Triton, ZMQ) with real‑robot demos on Franka and Oli.

  • Project
    nvidia-cosmos/cosmos-rl

    A scalable Reinforcement Learning framework specialized for Physical AI applications, providing a decoupled architecture for large-scale training workloads.

  • Project
    dexmal/dexbotic

    A PyTorch-based development toolbox for Vision-Language-Action (VLA) models that streamlines pretraining, fine-tuning, and evaluation for embodied intelligence.