Microduck with Isaac Lab 3.0: Training and Deploying Robot Policies from Simulation to Real Hardware

A key challenge in robotics is ensuring that policies trained in simulation perform reliably on real hardware; this workflow solves that by integrating Microduck with NVIDIA Isaac Lab 3.0, allowing developers to train robust walking policies in parallel simulation and deploy them directly onto the physical robot.

Overview of the Sim-to-Real Workflow

NVIDIA Isaac Lab is an open-source framework designed for training policies across thousands of GPU-parallel simulations, which is crucial for bridging the gap between simulated and real-world robotics. The Microduck robot, developed by Pollen Robotics, serves as the target hardware for this workflow. By combining Isaac Lab with Newton (an actuator modeling system) and implementing domain randomization, developers can train a walking policy in simulation, export it as an ONNX file, and deploy it onto a physical Microduck.

This process allows researchers to leverage the speed and safety of simulation while ensuring the resulting policy is robust enough to handle real-world factors like friction, mechanical delays, and servo play.

Setting Up the Training Environment

To begin the simulation and training process, specific hardware and software prerequisites must be met. For the simulation portion, a Linux workstation equipped with a supported NVIDIA GPU and driver is required. Software dependencies include Git and uv.

For the final deployment phase, a configured physical Microduck robot is necessary, along with its onboard software installed and a supported control interface, such as a paired gamepad.

Quick Installation Steps

Installation involves cloning the Isaac Lab repository and synchronizing its dependencies using uv. Once installed, users can confirm the availability of Microduck tasks and presets using the command uv run isaaclab list_envs --keyword MicroDuck --show_presets.

Microduck Environment Configuration

Isaac Lab provides a complete, ready-to-train environment for Microduck, including the robot model, scene, reward functions, and domain randomization capabilities. The initial task focuses on flat walking, with a rough variant available that introduces stairs, slopes, and uneven terrain.

Policy Interface Details

The walking policy interacts with the environment using a fixed interface: it accepts 61 observation values and outputs 14 joint positions. It is critical that the observation order, normalization, and action scaling remain consistent throughout the training, export, and deployment phases.

Accelerating Locomotion Training

Locomotion training is accelerated by Isaac Lab's ability to run multiple simulated environments in parallel on the GPU. Furthermore, distributed training across multiple GPUs can be utilized to increase training speed and policy robustness.

Modeling Real-World Dynamics

To ensure successful policy transfer from simulation, the training incorporates advanced physical modeling and randomization techniques:

  • Actuator Modeling: The system uses Newton with the MJWarp solver and Newton-based Better Actuator Models (BAM) to accurately model servo response. The backlash preset models the firmware controller, motor electrical limits, friction, and delay, converting position targets into realistic servo torques.
  • Domain Randomization: To prepare the policy for real hardware, various physical parameters are randomized for each Microduck instance. These randomized parameters include mass/center of mass, ground friction, joint inertia, encoder offsets, battery voltage/sag, and command latency. Terrain variation and external pushes are also applied.
  • Backlash Simulation: Backlash, defined as the slight movement in a gearbox before engagement when a joint changes direction, is modeled and varied across joints during training. This exposure helps the policy handle mechanical play encountered on the physical robot.

Training can be initiated using commands specifying the task (e.g., IsaacContrib-Velocity-Rough-MicroDuck), the number of environments (e.g., 4096), and the backlash preset.

Policy Evaluation and ONNX Export

Before deploying a policy to the physical robot, it must be evaluated in simulation and exported to the ONNX format. Isaac Lab's RSL-RL playback workflow handles this automatic export.

Evaluation Process

Evaluation is performed by running the latest checkpoint in the Newton visualizer. This allows users to observe the policy's performance across the simulated environments.

Exporting for Hardware Handoff

For deployment, the policy is exported as policy.onnx. This ONNX file is placed in an exported directory associated with the run. Microduck's onboard runtime is designed to load and execute these ONNX policies.

Deployment on Physical Microduck

Once the policy.onnx file is validated in simulation, it can be deployed to a configured Microduck robot using standard networking tools.

  1. Transfer: The policy file is copied from the workstation to the robot's host machine using scp.
  2. Loading: The policy is loaded onto the robot using the robotctl policy load walk command, replacing the default walking policy.
  3. Testing: The robot can then be tested using documented controls to verify standing, walking, turning, and stopping behavior. The deployed policy has demonstrated robustness, including the ability to recover footing after being knocked over.

Extending Capabilities with Isaac Lab 3.0

Isaac Lab 3.0 provides extensive tools to extend Microduck's capabilities beyond basic locomotion, enabling GPU-parallel learning across multiple domains.

  • Advanced Locomotion: Users can adapt the rough-terrain task configuration to incorporate curricula that gradually increase terrain difficulty, including stairs, slopes, and uneven ground.
  • Perception and Navigation: Policies can be extended by including head-camera depth and robot-state observations, allowing for perception-based tasks. Examples of vision-enabled manipulation are available in the Kuka–Allegro examples.
  • Dexterous Manipulation: The framework supports training arms and multi-fingered hands for manipulation tasks, such as the Kuka–Allegro lift and reorient tasks.
  • Custom Environments: Developers can utilize the project generator to build custom tasks and integrate SimReady assets from the SimReady Central Library to tailor observations and rewards for new challenges.

Community and Resources

Exported ONNX policies can be shared on the Hugging Face Hub or tested in the Microduck Arena, a Hugging Face Space designed for comparing compatible policies against shared challenges. This ecosystem encourages community contribution and validation of robotic AI models.

Sources