unilabsim/UniLab

UniLab: A Heterogeneous Architecture for Robot RL Beyond GPU-Dominant Paradigms

What it solves

UniLab addresses the limitation of GPU-dominant paradigms in robot reinforcement learning (RL). It provides a heterogeneous architecture that allows CPU-parallel simulation to run alongside GPU-accelerated policy training, removing the requirement for a GPU-based simulation backend.

How it works

UniLab uses a shared memory system (SharedReplayBuffer) to stream transitions from CPU-parallel physics simulations to GPU accelerators where policy learning occurs. It integrates multiple physics backends—specifically MuJoCoUni and MotrixSim—through backend-specific adapters and task owner configurations. This allows users to switch between simulation backends (e.g., from MuJoCo to MotrixSim) using simple configuration changes.

Who it’s for

It is designed for robotics researchers and developers working on reinforcement learning for embodied AI, particularly those who want to utilize diverse physics engines or optimize the distribution of simulation and training workloads across different hardware accelerators (CUDA, ROCm, XPU, or Apple Silicon).

Highlights

  • Heterogeneous RL Runtime: Decouples CPU-parallel simulation from GPU policy training via shared memory.
  • Multi-Backend Support: Integrated support for MuJoCoUni and MotrixSim.
  • Unified CLI: A single command-line interface for training and evaluating various RL algorithms including PPO, APPO, SAC, TD3, and FlashSAC.
  • Cross-Platform Compatibility: Supports Linux (CUDA, ROCm, XPU) and macOS (Apple Silicon).
  • Config-Driven Tasks: Uses Hydra YAML files to manage task, reward, backend, and algorithm settings.

Related

  • Project
    Tencent-Hunyuan/UniRL

    UniRL is a reinforcement learning framework that provides a unified post-training loop for multimodal models, including image and video diffusion, autoregressive VLMs, and LLMs.

  • Project
    RLinf/RLinf

    A flexible and scalable reinforcement learning infrastructure designed to streamline the training of Embodied and Agentic AI across diverse hardware and simulators.

  • Project
    vikashplus/robohive

    A unified framework for robot learning that provides a collection of MuJoCo-simulated environments and tasks exposed via the OpenAI-Gym API.

  • Project
    mani-skill/ManiSkill

    An open-source robot simulation and training framework that uses GPU parallelization to enable high-throughput data collection and training for manipulation skills.

  • Project
    nvidia-cosmos/cosmos-rl

    A scalable Reinforcement Learning framework specialized for Physical AI applications, providing a decoupled architecture for large-scale training workloads.