X-Square-Robot/wall-x

Building General-Purpose Robots Based on Embodied Foundation Model

What it solves

Wall-X provides a framework for building general-purpose robots using embodied foundation models. It aims to bridge the gap between vision-language models (VLMs) and physical interaction, allowing robots to understand how to act effectively within the physical world based on high-fidelity interaction data.

How it works

The project implements a Vision-Language-Action (VLA) architecture. It leverages large-scale multimodal pretraining to associate language and vision with specific robotic actions. The system includes specialized training and inference stacks, including flow-matching and FAST action branches, and utilizes CUDA operators for performance. It integrates with the LeRobot dataset format for data preparation and supports fine-tuning via FSDP (Fully Sharded Data Parallel) training.

Who it’s for

This is designed for robotics researchers and developers working on embodied AI, specifically those looking to deploy and fine-tune Vision-Language-Action models for real-robot manipulation and simulator evaluation.

Highlights

  • Deployment-Ready VLA: Includes the Wall-OSS-0.5 model for direct real-robot manipulation.
  • World Action Modeling: Features WALL-WM, which couples future-video imagination with action prediction.
  • Integrated Ecosystem: Provides tools for LeRobot data preparation, WebSocket serving for real-time action inference, and LIBERO simulator evaluation.
  • Performance Optimized: Uses FlashAttention and custom CUDA operators to ensure efficient inference and training.

Related

  • Project
    dexmal/opendm

    OpenDM is a Vision-Language-Action (VLA) model framework for open-world robot control, providing tools for training, fine-tuning, and deploying models that can handle complex, multi-embodiment tasks.

  • Project
    dexmal/dexbotic

    A VLA (Vision-Language-Action) development toolbox for embodied intelligence that provides a unified framework for pretraining, fine-tuning, and evaluating robotic policies.

  • Project
    2toinf/X-VLA

    X-VLA is a soft-prompted Transformer-based Vision-Language-Action model designed for scalable cross-embodiment robotic control across diverse hardware platforms.

  • Project
    Robbyant/lingbot-vla

    A pragmatic Vision-Language-Action (VLA) foundation model pre-trained on 20,000 hours of real-world robot data to enable high-performance robotic control across multiple platforms.

  • Project
    BeingBeyond/Being-H

    A family of human-centric embodied foundation models, including VLA and WAM architectures, designed to enable robots to learn and generalize actions from human videos.