PKU-EPIC/GraspVLA

What it solves

GraspVLA addresses the high cost and data scarcity associated with training robotic grasping models. It enables robots to perform open-vocabulary grasping of diverse objects in the real world without requiring expensive real-world fine-tuning, achieving direct sim-to-real transfer.

How it works

The project consists of three main components:

1. SynGrasp-1B: A massive synthetic dataset containing one billion frames of grasping data across 240 object categories and over 10,000 objects.
2. GraspVLA Model: A Vision-Language-Action (VLA) model pretrained on the synthetic data. It uses a Unified Chain-of-Thought (CoT) framework that combines autoregressive perception (predicting bounding boxes and goals) with flow-matching-based action generation.
3. Hybrid Training: The model is trained on both synthetic action data and internet-scale semantic data, allowing it to understand and act upon open-vocabulary commands.

Who it’s for

Robotics researchers and developers working on robotic manipulation, sim-to-real transfer, and foundation models for embodied intelligence.

Highlights

• Zero-Shot Generalization: Capable of grasping real-world objects without additional fine-tuning after synthetic pretraining.
• Billion-Scale Data: Leverages the SynGrasp-1B dataset to provide a vast diversity of training examples.
• Efficient Inference: Achieves 200ms inference latency using approximately 9GB of GPU memory on an NVIDIA RTX L40s.
• Integrated Reasoning: Combines perception and action into a single reasoning process via a CoT framework.