Grabette: An Open System for Robot-Manipulation Data Collection

Hugging Face and Pollen Robotics have introduced Grabette, an open-source, low-cost system designed to record robot-manipulation data. By allowing humans to record demonstrations using a handheld gripper rather than teleoperating a physical robot, Grabette aims to eliminate the data bottleneck in robot learning and seed a large, open, collaborative manipulation dataset.

Solving the Robot Learning Data Bottleneck

Robot learning currently faces a supply problem where capable policy architectures—such as transformer-based VLAs, diffusion policies, and world models—outpace the availability of large, diverse, real-world manipulation data. Traditional teleoperation is often expensive, demanding, and difficult to scale across various environments because it requires a physical robot and complex hardware rigs.

Grabette addresses this by enabling data collection without a robot. Users can capture 6-DoF trajectories of human hand movements using a handheld gripper, which provides data that a robot can then learn from. This approach lowers the barrier to entry, allowing anyone with the hardware to contribute to a shared dataset.

Technical Architecture and Hardware

Grabette is inspired by the Universal Manipulation Interface (UMI) and is designed to be robot-agnostic. It utilizes a dual-camera system to separate observation and tracking roles:

• Observation Camera: A low-cost wide fisheye lens provides the context-rich, wrist-camera-style view required for the policy.
• Tracking Camera: An RGBD camera handles the robust 6-DoF tracking of the demonstration.

Hardware Components and Costs

The system consists of two primary hardware components, both of which are open-source (CAD and production files available on GitHub):

1. Grabette (Handheld Device): Includes a camera, IMU, and gripper. It has a Bill of Materials (BOM) cost of approximately 490⁄.
2. Gripette (Robot Gripper): A motorized end-effector twin of Grabette, featuring a camera and two servomotors, with a BOM cost of approximately 120⁄. This is used to execute the learned movements on a real or simulated robot arm.

The device is built using standard, off-the-shelf components, including a Raspberry Pi, a standard Pi camera, an OAK-D depth camera, and magnetic encoders.

Data Collection and Processing Pipeline

Grabette simplifies the transition from demonstration to training-ready datasets in two steps:

1. Recording

Users record tasks by pressing a button on the device. The system simultaneously records data from the observation camera, the tracking camera (color, depth, and IMU), and the gripper’s encoder joint values, all synchronized by a single shared clock. Data is saved locally on the Raspberry Pi.

2. Browser-Based Processing

Post-processing is handled via a browser-based dashboard and a Hugging Face Space, requiring no local installation:

• Upload: Selected episodes are uploaded to the Hugging Face Hub.
• SLAM: The grabette-slam space uses the RTAB-MAP library to perform SLAM and verify trajectory correctness (checking for jumps or loss of tracking).
• Conversion: Episodes are converted into the LeRobot format.
• Visualization: The final dataset is uploaded to a user's space for viewing via the LeRobot visualizer.

End-to-End Implementation Example

To demonstrate the utility of the system, the developers provided an example where 200 recorded demonstrations were used to train a Diffusion Policy (ResNet18 + SpatialSoftmax encoder, DDIM scheduler, 6-D rotation actions) using the LeRobot stack. This policy was then evaluated on an OpenArm 7-DoF arm equipped with the Gripette gripper, driven over a gRPC API.

Future Developments

Beyond the current release, the project intends to expand its capabilities. A work-in-progress device called Casquette, a head-mounted POV device, is being developed to complement Grabette for egocentric capture.