Qwen-RobotManip: Alignment Unlocks Scale for Robotic Manipulation Foundation Models

Qwen-RobotManip is a generalizable Vision-Language-Action (VLA) foundation model that demonstrates that scaling robotic manipulation data is only effective when paired with a unified alignment framework. By aligning representations across different robot embodiments, sensors, and tasks, the model achieves superior generalization to novel scenes, unseen instructions, and cross-embodiment transfers using over 38,100 hours of open-source and synthesized data.

Unified Cross-Embodiment Alignment Framework

Qwen-RobotManip solves the problem of conflicting signals during large-scale multi-source training by introducing a three-dimensional alignment across representation, motion, and behavior.

Canonical State-Action Representation

All robot states and actions are mapped to a unified 80-dimensional vector. This vector accommodates single-arm, dual-arm, dexterous hand, and mobile base configurations. To prevent conflicts between different embodiments, a per-dimension binary mask ensures that gradients only flow through the populated slots relevant to the specific robot being trained.

Camera-Frame Delta Pose

To make visually similar motions numerically proximate across different robot platforms, end-effector actions are expressed as deltas in the camera coordinate frame rather than the robot base frame. The model utilizes Camera Positional Encoding (CaPE) in cross-attention layers for camera extrinsics and encodes intrinsics into visual tokens for field-of-view awareness.

In-Context Policy Adaptation

The model employs a structured embodiment prompt—specifying the robot platform, execution speed, and FPS—alongside a historical observation-action chunk. This allows Qwen-RobotManip to adapt its policy on-the-fly to different embodiments and behavior patterns. A stochastic context sampling strategy is used during training to prevent the model from relying on action-copy shortcuts.

Human-to-Robot Data Synthesis at Scale

Because robotic manipulation data is scarce, Qwen-RobotManip utilizes a synthesis pipeline to convert egocentric human manipulation videos into robot demonstrations.

The Pipeline: The system converts 1,933 hours of egocentric human video into 24,808 hours of robot demonstrations across 15 different embodiments. This is achieved through action retargeting, hand removal and inpainting, simulated rendering, and depth-guided compositing.
Total Corpus: The resulting pretraining corpus totals approximately 38,100 hours, consisting of robot data (~~11,420h), egocentric human data (~~1,933h), and synthesized human-to-robot data (~24,808h).
Curation: A multi-stage pipeline ensures quality via five state-action filtering stages (removing noisy actions and verifying kinematic consistency) and three cross-modal checks to validate that language instructions match video content and visual observations match robot states.

Model Architecture and Training

Qwen-RobotManip couples a Qwen3.5-4B vision-language backbone with a flow-matching Diffusion Transformer (DiT) action head.

Training is conducted in two primary phases:

Pre-training: A dual-stream co-training approach is used, combining a VLA stream (robot manipulation data) and a VLM stream (vision-language understanding data) at a 9:1 ratio.
Post-training: The model undergoes generalist SFT (Supervised Fine-Tuning) on all demonstration data for each benchmark, co-training with VL and VLA data to improve OOD instruction following.

Out-of-Distribution (OOD) Generalization Performance

Qwen-RobotManip prioritizes OOD benchmarks as the primary metric for success, arguing that in-distribution (IID) scores can be achieved through pattern matching rather than genuine generalization.

Simulation Benchmarks

LIBERO-Plus: Qwen-RobotManip-Context achieves 91.4% overall success, significantly outperforming $\pi_{0.5}$ (84.4%). It shows particular strength in handling camera and robot perturbations.
RoboTwin-C2R Hard: The model achieves a 69.4% success rate on "Hard" environmental randomizations, which is 21.5% higher than $\pi_{0.5}$.
RoboCasa365: On the "Composite-Unseen" task (long-horizon tasks in OOD scenes), the model achieves 14.9%, nearly three times the performance of the next-best model (5.4%).
EBench: The model achieves a 45.6% overall success rate, outperforming $\pi_{0.5}$ (27.1%) and other baselines.
RoboTwin-IF: In instruction following with unseen templates, the model achieves 72.2% average success, a 22.6 point lead over $\pi_{0.5}$.
RoboTwin-XE: In zero-shot cross-embodiment transfer (trained on AgileX ALOHA, tested on unseen robots), the camera-frame EEF representation allows the model to reach 23.9% success, 3.2x that of $\pi_{0.5}$ (eef).

Real-World Evaluation

Novel Scenes and Instructions: In-domain success was 88.6% and OOD success was 87.5%, compared to $\pi_{0.5}$'s 37.5% OOD success.
Few-Shot Adaptation: When fine-tuned on only 130 demonstrations across 5 tasks, Qwen-RobotManip outperformed baselines on 4 of 5 tasks.
Cross-Embodiment Skill Transfer: A policy fine-tuned on CobotMagic and ARX data achieved 55.0% success on novel ARX tasks with zero training demonstrations, 4x better than ablation variants without the unified framework.
RoboChallenge Table30 v1: Qwen-RobotManip ranked 1st in the generalist track with a 45% success rate, leading the third-place finisher by 20%.

Key Findings on Scaling and Recovery

Alignment as a Prerequisite: The research found that only models with unified cross-embodiment representations exhibit clean log-linear data scaling. Without alignment, adding more data leads to erratic or flat scaling curves.
Bimanual Coordination: The model achieved 40% success on bimanual coordination tasks, significantly higher than $\pi_{0.5}$ (21.2%). It was the only model to succeed on the "pour fries into plate" task (30% success).
Emergent Recovery: The model demonstrates reactive error recovery (e.g., autonomously retrying a grasp after a slip), a behavior that emerged from large-scale pretraining rather than explicit programming.