fan-ziqi/rl_sar

Simulation verification and physical deployment of robot reinforcement learning algorithms, suitable for quadruped robots, wheeled robots, and humanoid robots. "sar" represents "simulation and real"

What it solves

rl_sar provides a unified framework for the simulation verification and physical deployment of reinforcement learning (RL) policies on various robots. It bridges the gap between simulation and reality ("sim-to-real") by supporting a wide array of robot hardware, simulators, and inference engines, allowing developers to test RL-based locomotion and dance skills in simulation before deploying them to physical hardware.

How it works

The framework acts as a deployment layer that takes pre-trained RL policies (in .pt or .onnx formats) and executes them on the robot. It supports multiple simulation environments (Gazebo and MuJoCo) and multiple communication protocols (ROS Noetic, ROS2 Foxy/Humble, and standalone CMake builds for hardware deployment).

Key technical components include:

  • Inference Engines: Uses libtorch and onnxruntime to run the neural networks.
  • Robot Support: Provides pre-configured support for various quadruped, wheeled, and humanoid robots (e.g., Unitree A1, Go2, G1, Agibot D1).
  • Actuator Network Training: Includes tools to log real-world motor data and train an actuator network to better model the physical robot's behavior.
  • Control Interfaces: Supports gamepad, keyboard, and experimental mobile web control.

Who it’s for

Robotics researchers and engineers working with reinforcement learning who need a standardized way to deploy trained policies to physical robots or verify them in simulation across different hardware platforms.

Highlights

  • Broad Hardware Compatibility: Supports quadrupeds, wheeled robots, and humanoids.
  • Multi-Simulator Support: Compatible with IsaacGym, IsaacSim, Gazebo, and MuJoCo.
  • Flexible Deployment: Supports both ROS-based and non-ROS (CMake) builds for direct hardware execution.
  • Cross-Platform: Works on Linux and macOS (MuJoCo only).
  • Sim-to-Real Tooling: Includes a dedicated pipeline for training actuator networks using real-world data.

Related

  • Project
    nvidia-cosmos/cosmos-rl

    A scalable Reinforcement Learning framework specialized for Physical AI applications, providing a decoupled architecture for large-scale training workloads.

  • Project
    ARISE-Initiative/robosuite

    A MuJoCo-powered simulation framework and benchmark suite for robot learning, providing modular tools to create environments and test AI control algorithms.

  • Project
    vikashplus/robohive

    A unified framework for robot learning that provides a collection of MuJoCo-simulated environments and tasks exposed via the OpenAI-Gym API.

  • Project
    RLinf/RLinf

    A flexible and scalable reinforcement learning infrastructure designed to streamline the training of Embodied and Agentic AI across diverse hardware and simulators.

  • Project
    NVlabs/ProtoMotions

    A GPU-accelerated simulation and learning framework for training physically simulated digital humans and humanoid robots, supporting large-scale motion learning and sim-to-real deployment.