VLARLKit/VLARLKit

Elegant VLA-RL library

What it solves

VLARLKit provides a researcher-friendly framework for training Vision-Language-Action (VLA) models using Reinforcement Learning (RL). It addresses the common problem of dependency conflicts between base models, RL training stacks, and robot simulation environments by decoupling them into separate projects and processes.

How it works

The library uses a modular architecture that separates the policy, rollout, runner, and model layers. To prevent software conflicts, it employs a dependency-decoupled design where model backends are managed as separate projects and benchmark environments run as independent ZMQ processes. It supports both on-policy and off-policy training, including asynchronous off-policy training to allow data collection to happen simultaneously with model updates.

Who it’s for

It is designed for AI researchers working on embodied intelligence and VLA models who need a flexible, easy-to-extend PyTorch library for RL experimentation.

Highlights

  • Decoupled Architecture: Separates base-model and simulator dependencies to avoid installation conflicts.
  • Broad Algorithm Support: Includes on-policy (PPO, GRPO), off-policy (DSRL, RLT), and model-based RL (VLA-MBPO).
  • VLA Model Integration: Supports flow-based models like π†.₅ and autoregressive models like OpenVLA-OFT.
  • Simulation Benchmarks: Integrated support for LIBERO, ManiSkill, and RoboTwin.

Related

  • Project
    yihaosun1124/OfflineRL-Kit

    A PyTorch library for offline reinforcement learning that provides a standardized framework and implementations of state-of-the-art model-free and model-based algorithms.

  • Project
    meta-pytorch/torchforge

    A PyTorch-native agentic RL library that separates infrastructure concerns from model concerns to simplify scalable reinforcement learning experimentation.

  • Project
    dexmal/dexbotic

    A VLA (Vision-Language-Action) development toolbox for embodied intelligence that provides a unified framework for pretraining, fine-tuning, and evaluating robotic policies.

  • Project
    pykt-team/pykt-toolkit

    A PyTorch-based library for training and benchmarking deep learning knowledge tracing models with standardized data preprocessing and multiple model implementations.

  • Project
    huggingface/lerobot

    A PyTorch-native library providing models, datasets, and tools to standardize real-world robotics, lowering the barrier to entry for training and deploying AI policies.