Open-X-Humanoid/HEX

HEX is a whole-body vision-language-action framework for full-sized humanoid robots.

What it solves

HEX addresses the challenge of performing complex, long-horizon whole-body manipulation across different humanoid robot platforms. It solves the problem of "embodiment gap"—where data from one robot cannot be easily used by another—by aligning diverse humanoid states into a shared format, allowing a single policy to transfer across different hardware.

How it works

HEX is a vision-language-action (VLA) framework that uses a Qwen-VL backbone for visual and linguistic understanding. It employs two specialized components to handle robot movement:

  1. Unified Proprioceptive Predictor (UPP): Aligns heterogeneous humanoid body states into shared "body-part slots."
  2. Flow-matching action head: Predicts continuous future actions for the robot's arms, hands, and waist.

For deployment, HEX handles the high-level coordination of the upper body and waist, while sending high-level commands to a separate low-level RL-based controller that manages the stability and movement of the legs.

Who it’s for

Researchers and developers working on humanoid robotics, specifically those focused on cross-embodiment learning, whole-body control, and vision-language-action models.

Highlights

  • Cross-Embodiment Transfer: Learns predictive body dynamics from data across multiple humanoid platforms (e.g., Unitree G1, H1, Tiangong series).
  • Whole-Body Coordination: Integrates vision, language, and proprioception to coordinate arm, hand, and waist movements.
  • Extensive Datasets: Provides processed multi-embodiment pretraining datasets and real-world task datasets.
  • Hybrid Control Architecture: Combines a high-level VLA policy with low-level RL controllers for stable humanoid manipulation.

Related

  • Project
    X-Square-Robot/wall-x

    An open-source framework for embodied foundation models that enables robots to perform complex physical manipulations by bridging vision-language understanding with action prediction.

  • Project
    2toinf/X-VLA

    X-VLA is a soft-prompted Transformer-based Vision-Language-Action model designed for scalable cross-embodiment robotic control across diverse hardware platforms.

  • Dispatch
    Qwen-VLA: Unifying Vision-Language-Action Modeling for Embodied Intelligence

    Qwen-VLA is a general-purpose Vision-Language-Action model that unifies robotic manipulation, vision-language navigation, and cross-embodiment control into a single generalist policy model.

  • Project
    MINT-SJTU/Evo-1

    Evo-1 is a lightweight Vision-Language-Action (VLA) model for robotic manipulation that translates visual and textual instructions into robot actions across various simulations and real hardware.

  • Project
    physical-superintelligence-lab/Psi0

    An open vision-language-action (VLA) model for humanoid robots that enables dexterous loco-manipulation by combining a VLM backbone with a diffusion transformer action expert.