Open-X-Humanoid/HEX

HEX is a whole-body vision-language-action framework for full-sized humanoid robots.

What it solves

HEX addresses the challenge of whole-body manipulation for full-sized humanoid robots across different hardware platforms. It enables robots to perform long-horizon manipulation tasks by learning from heterogeneous data sources and transferring policies across different humanoid embodiments.

How it works

HEX is a vision-language-action (VLA) framework that uses a Qwen-VL backbone for visual and linguistic understanding. It employs a Unified Proprioceptive Predictor (UPP) to align different humanoid states into shared body-part slots, allowing the model to learn predictive body dynamics from diverse datasets. A flow-matching action head predicts continuous future actions for the arms, hands, and waist. For stability, HEX provides high-level commands to a low-level RL-based whole-body controller that handles the leg actions.

Who it’s for

This project is for robotics researchers and developers working on humanoid robot control, cross-embodiment learning, and vision-language-action models.

Highlights

  • Cross-Embodiment Capability: Aligns heterogeneous humanoid states to enable policy transfer across different platforms (e.g., Unitree G1, H1, Leju Kuavo).
  • Whole-Body Coordination: Combines high-level VLA predictions for upper body and waist with low-level RL controllers for leg stability.
  • Comprehensive Data Support: Includes pretraining and fine-tuning code for datasets from various humanoid platforms.
  • Simulation Support: Can be evaluated and trained in simulation environments like LIBERO.

Related

  • Project
    NVIDIA/Isaac-GR00T

    An open vision-language-action (VLA) model for generalized humanoid robot skills, enabling cross-embodiment manipulation tasks via multimodal input.

  • Project
    2toinf/X-VLA

    X-VLA is a soft-prompted Transformer-based Vision-Language-Action model designed for scalable cross-embodiment robotic control across diverse hardware platforms.

  • Dispatch
    Qwen-VLA: Unifying Vision-Language-Action Modeling for Embodied Intelligence

    Qwen-VLA is a general-purpose Vision-Language-Action model that unifies robotic manipulation, vision-language navigation, and cross-embodiment control into a single generalist policy model.

  • Project
    dexmal/opendm

    OpenDM is a Vision-Language-Action (VLA) model framework for open-world robot control, providing tools for training, fine-tuning, and deploying models that can handle complex, multi-embodiment tasks.

  • Project
    X-Square-Robot/wall-x

    An open-source framework for embodied foundation models that enables robots to perform complex physical manipulations by bridging vision-language understanding with action prediction.