pollen-robotics/microduck_rl

RL training environments for Microduck (mjlab)

What it solves

This project provides reinforcement learning (RL) training environments to develop complex motor skills for the Microduck, a small bipedal robot. It specifically addresses the "sim2real gap"—the difficulty of transferring behaviors learned in simulation to a physical robot—by using high-fidelity actuator models and domain randomization.

How it works

The system uses MuJoCo Warp and PPO (Proximal Policy Optimization) to train policies at 50 Hz. To ensure these policies work on the real robot, it incorporates:

  • BAM Actuator Physics: Models the Dynamixel XL330 servos down to the voltage control law, including back-EMF and friction.
  • Domain Randomization: Randomizes battery voltage, voltage sag, command delay, and friction during training.
  • Backlash Simulation: Includes specific training variants that model gear play (±1° per joint) to make the robot more robust to mechanical imperfections.
  • ONNX Export: Trained policies are exported to ONNX format with observation normalizers baked in, allowing them to be deployed directly to the robot's runtime.

Who it’s for

  • Robotics researchers and developers working on bipedal locomotion.
  • Users of the Microduck robot who want to train custom gaits or tricks.
  • Engineers interested in sim2real transfer techniques for small-scale robotics.

Highlights

  • Diverse Task Library: Includes environments for walking (flat and rough terrain), fall recovery, standing up, sitting, ball kicking, and even roller-skating.
  • Hot-Swappable Policies: All policies share a 61-dimensional observation contract, allowing the robot to switch between walking, recovering, and performing tricks seamlessly.
  • Simplified Deployment: A publish tool allows users to upload policies to the Hugging Face Hub for easy installation on the physical robot via a CLI tool.
  • Hardware-Aware Training: Specifically models actuator voltage and load-dependent friction rather than using ideal PD controllers.

Related

  • Project
    AI-FanGe/Microduck-build-tutorial

    A compact biped robot project providing a full pipeline from MuJoCo-based RL training to real-world deployment on a Raspberry Pi Zero 2 W using ONNX policies.

  • Dispatch
    Pollen Robotics Microduck: An Open-Source Biped for RL Training

    Pollen Robotics has launched Microduck, a 25cm open-source biped robot designed for reinforcement learning (RL) training via a sim-to-real pipeline using MuJoCo.

  • Project
    pollen-robotics/microduck

    Microduck is the control software for a tiny biped robot that uses reinforcement learning policies to walk, roll, and interact with its environment.

  • Project
    Farama-Foundation/Gymnasium-Robotics

    A collection of Reinforcement Learning robotic environments powered by the MuJoCo physics engine and the Gymnasium API for training AI agents in manipulation and navigation tasks.

  • Project
    fan-ziqi/rl_sar

    A framework for simulation verification and physical deployment of reinforcement learning policies for quadruped, wheeled, and humanoid robots.