physical-superintelligence-lab/Psi0

What it solves

Psi0 is an open vision-language-action (VLA) model designed for dexterous humanoid loco-manipulation. It addresses the challenge of enabling humanoid robots to perform complex, long-horizon tasks that require both movement (locomotion) and precise hand manipulation (dexterity) by leveraging large-scale human video data and a small amount of robot-specific data.

How it works

Psi0 uses a tiered architecture to translate visual and linguistic instructions into physical actions:

• System-2 (Vision-Language Backbone): Based on Qwen3-VL-2B-Instruct, this component extracts high-level semantic features from visual observations and text instructions.
• System-1 (Action Expert): A multimodal diffusion transformer (approximately 500M parameters) that takes the features from System-2 and predicts future whole-body action chunks.
• System-0 (Tracking Controller): An RL-based controller that executes the predicted lower-body commands to ensure stable physical movement.

The model is first pre-trained on large-scale human egocentric videos to learn task semantics and visual representations, then post-trained on real-world teleoperated robot data to learn the specific dynamics of the robot's body.

Who it’s for

This project is intended for robotics researchers and developers working with humanoid robots (specifically the Unitree G1) and those interested in VLA models for whole-body control.

Highlights

• Efficient Fine-tuning: Capable of acquiring new dexterous loco-manipulation skills with as few as 80 trajectories.
• Hybrid Architecture: Combines a VLM backbone with a diffusion transformer for action prediction.
• Integration: Supports integration with SONIC, a powerful whole-body controller for humanoid robots.
• Simulation Support: Includes benchmarking via the SIMPLE simulator (MuJoCo physics and Isaac Sim rendering).
• Open Assets: Provides open-source checkpoints and datasets on Hugging Face.