Robbyant/lingbot-vla-v2

What it solves

LingBot-VLA 2.0 is a Vision-Language-Action (VLA) foundation model designed to bridge the gap between large-scale pre-training and reliable real-world robot applications. It addresses the challenge of generalizing across different robot types (embodiments) and tasks while maintaining high performance in both simulation and real-world environments.

How it works

The model utilizes a large-scale pre-training corpus of 60,000 hours of data, including 50,000 hours of robot trajectories and 10,000 hours of egocentric human videos. It employs several key technical innovations:

• Unified Action Representation: It maps diverse robot configurations into a 55-dimensional canonical state/action vector covering arms, end-effectors, grippers, dexterous hands, waist, head, and mobile bases.
• MoE Action Expert: Sparse Mixture-of-Experts (MoE) layers are used within the action expert to allow universal priors and specialized embodiment/task patterns to coexist.
• Dual-Query Distillation: The model distills perceptual queries from LingBot-Depth and DINO-Video to capture both current scene geometry and future scene evolution (predictive dynamics modeling).

Who it’s for

This project is for robotics researchers and developers who want to deploy generalist robot policies that can work across multiple hardware configurations and perform complex manipulation tasks.

Highlights

• Broad Generalization: Trained on a massive dataset across 20 different robot configurations.
• Expanded Action Space: Supports a wide range of robot components beyond standard dual-arm manipulation.
• High Performance: Outperforms previous versions and competitors on GM-100 bimanual manipulation and long-horizon mobile manipulation benchmarks.
• Real-world Ready: Includes deployment scripts for both simulation (RoboTwin) and real-world robots, with inference taking approximately 130ms on an RTX 4090D.