mani-skill/ManiSkill

What it solves

ManiSkill provides a high-performance simulation framework for training robots in manipulation tasks. It addresses the bottleneck of slow data collection and simulation by leveraging GPU parallelization for both physics and visual rendering, allowing researchers to train embodied AI models much faster than traditional CPU-based simulators.

How it works

Powered by SAPIEN, the framework enables GPU-parallelized simulation and visual data collection (RGBD and segmentation). It supports heterogeneous simulation, meaning multiple parallel environments can each have different scenes and objects. It also provides a task-building API that simplifies GPU memory management and includes tools for both Real2sim (evaluating real-world policies in simulation) and Sim2real (deploying simulated policies to physical robots).

Who it’s for

It is designed for researchers and developers working on robot learning, specifically those focusing on manipulation skills using Reinforcement Learning, Imitation Learning, or Vision Language Action (VLA) models.

Highlights

• Extreme Throughput: Capable of collecting visual data at over 30,000 FPS on a single NVIDIA 4090 GPU.
• Diverse Embodiments: Supports various robots including humanoids, mobile manipulators, and single-arm robots.
• Broad Task Range: Includes environments for table-top tasks, drawing, cleaning, and dexterous manipulation.
• Integrated Baselines: Comes with tuned baselines for PPO, SAC, Diffusion Policy, and large VLA models like Octo and RDT-1B.