NVlabs/mimicgen

This code corresponds to simulation environments used as part of the MimicGen project.

What it solves

Training robots to perform tasks in the real world normally requires a huge number of human demonstrations, which are slow and expensive to collect. MimicGen tackles this by generating many new task demonstrations from a small set of human examples, so robot learning can scale without needing humans to record every variation.

How it works

MimicGen provides code that takes human demonstrations of a task and uses simulation to synthesize additional demonstrations with different object placements, robot start positions, and other variations. It includes ready-to-use simulation environments and pre-generated datasets. The released datasets contain over 48,000 demonstrations across 12 tasks, and the tool can create as many more as you need. The system is based on the CoRL 2023 paper of the same name.

Who it's for

Researchers and developers working on robot learning, especially those who want to train policies with large amounts of varied demonstration data without physically collecting each one. It's also useful for people studying scalable data generation for robotic manipulation.

Highlights

  • Over 48,000 demonstrations already released across 12 different tasks
  • Data generation tool can create unlimited additional demonstrations
  • Includes simulation environments and full data generation code
  • Official release from NVIDIA Labs, with documentation and tutorials for getting started, custom environments, and reproducing published results
  • Datasets are released under the permissive CC-BY 4.0 license

Related

  • Project
  • Project
  • Project
  • Project