NVlabs/GraspGen

What it solves

GraspGen is a framework for 6-DOF robotic grasping that allows robots to determine how to pick up objects in diverse environments. It addresses the challenge of generating high-quality grasp poses across different gripper types, varying levels of 3D observability (partial vs. complete point clouds), and different scene complexities, ranging from single objects to cluttered environments.

How it works

The system uses a diffusion-based framework to generate 6-DOF grasp candidates. It employs a modular architecture that can be adapted to different embodiments (such as industrial pinch grippers or suction cups). To ensure the quality of the generated grasps, it uses a grasp discriminator that scores and ranks the candidates. The framework supports various input formats, including segmented object point clouds, 3D meshes (OBJ, STL, PLY, USD), and full scene point clouds with optional collision filtering to ensure the robot does not collide with the environment.

Who it’s for

This project is designed for robotics researchers and developers building grasping systems for industrial or research robots (e.g., Franka Panda, Robotiq-2f-140). It is also suitable for those integrating robotic grasping into LLM-driven agents via tool-calling interfaces.

Highlights

• Cross-Embodiment Support: Works with multiple gripper types, including pinch grippers and suction cups.
• High Performance: Achieves state-of-the-art performance on the FetchBench benchmark (17% improvement) while being 21x more memory-efficient.
• Real-time Inference: Capable of running at 20 Hz before TensorRT optimization.
• LLM Integration: Includes a Model Context Protocol (MCP) and ZMQ-based server to allow LLMs to call grasping as a tool.
• Robustness: Handles both partial and complete 3D point clouds and supports collision-free grasp generation in cluttered scenes.