open-gigaai/giga-brain-0

GigaBrain-0: A World Model-Powered Vision-Language-Action Model

GigaBrain‑0.7 – An Embodied Foundation Model

What it is – GigaBrain‑0.7 is a large‑scale vision‑language‑action (VLA) model designed to control many different robot platforms. It follows a three‑system architecture that jointly learns visual understanding, language grounding, and robot‑specific action generation. The model has been pretrained on ≈ 37 000 h of heterogeneous embodied data (real‑robot runs, simulation, egocentric video, world‑model‑generated trajectories, etc.) and is released in a 3.5 B‑parameter base version plus several fine‑tuned variants.

Why it matters – Most existing VLA systems are trained on a single robot type or on relatively small datasets. GigaBrain‑0.7 shows that a single model can generalise across many embodiments (e.g., AgileX cobot, Franka, dexterous‑hand rigs, the Maker H01 platform) while retaining strong zero‑shot performance on new tasks. The project also ships a full training stack, evaluation benchmarks, and ready‑to‑run deployment servers, making it one of the most complete open‑source embodied‑AI releases to date.

Quick‑start Overview

  1. Set up the environment – Create a fresh Conda env (Python 3.11) and install the pinned dependencies (giga-datasets, giga-train, giga-models) plus the requirements-paligemma2.txt stack.
  2. Download a model – Grab the 3.5 B base checkpoint (or a fine‑tuned variant) from the Hugging Face links in the README.
  3. Prepare data – Convert your robot logs to the LeRobot Parquet format, then run scripts/compute_norm_stats_fast.py to generate normalisation statistics for the specific robot type.
  4. Train – Choose one of the two provided configs (gb07_pg2_pick_and_place_piper_30k.py for AgileX PiPER or gb07_pg2_push_buttons_h01_30k.py for Maker H01) and launch scripts/train.py.
  5. Evaluate offline – Use scripts/inference/inference_gigabrain07_flow_rollout.py to run the trained checkpoint on saved LeRobot episodes and obtain MSE/MAE metrics.
  6. Deploy – Start the unified inference server (inference_agilex_server_unified.py or the H01 server script) and connect a ROS client that streams camera images and receives joint‑space commands.

Core Components

Component Role
Three‑system architecture Separate modules for perception, language grounding, and action generation that share a common backbone.
Embodiment IDs Integer codes (0‑7) that select robot‑specific state/action projections; new robots can be added by extending the mapping.
LeRobot data format Standardised Parquet schema for multi‑camera, joint‑state, and action streams; the repo provides scripts to compute normalisation stats.
Training stack giga-datasets (data loading), giga-train (training loops, optimisers), giga-models (model definitions built on the Paligemma‑2 backbone).
Benchmarks VLM‑Benchmark, RoboColiseum, RoboTwin 2.0, and EBench – each with its own README and evaluation scripts.
Deployment server Diffusers‑compatible checkpoint that loads the training‑time transforms and serves predictions over a simple TCP/ROS interface.

Benchmarks & Evaluation

Benchmark What it tests How to run
VLM‑Benchmark Multi‑modal (vision‑language‑action) evaluation across a suite of embodied tasks. Follow scripts/eval/VLM-Benchmark/README.md.
RoboColiseum Post‑training instruction following, spatial reasoning, and manipulation on a simulated arena. See configs/robocoliseum_posttrain_release/README.md.
RoboTwin 2.0 50 bimanual tasks in simulation, measuring transfer from pre‑training to new manipulation problems. Follow configs/robotwin_posttrain_release/eval_robotwin/README.md.
EBench General‑purpose manipulation (GenManip) evaluation, both locally and on an online server. See configs/ebench_posttrain_release/eval_ebench/README.md.

License & Citation

  • License: Apache 2.0 (permissive, commercial‑friendly).
  • Citation: Use the provided BibTeX entry (arXiv 2608.15875) when referencing the model or the technical report.

Who might use this?

  • Robotics researchers wanting a large‑scale pretrained VLA model to fine‑tune on a new robot.
  • Industrial labs looking for a ready‑to‑deploy inference server that can be wrapped in ROS.
  • Educators who need a concrete example of a multi‑embodiment foundation model and its full training/evaluation pipeline.

Bottom line: GigaBrain‑0.7 is a genuine, end‑to‑end open‑source embodied AI project that supplies data, model weights, training code, benchmark suites, and deployment tools for vision‑language‑action robotics.

Related

  • Project
    alibaba-damo-academy/RynnBrain

    An embodied foundation model family that bridges vision-language understanding with real-robot control through 3D grounding and contact point prediction.

  • Project
    X-Square-Robot/wall-x

    An open-source framework for embodied foundation models that enables robots to perform complex physical manipulations by bridging vision-language understanding with action prediction.

  • Project
    BeingBeyond/Being-H

    A family of human-centric embodied foundation models, including WAM and VLA models, designed for cross-embodiment generalization and latent reasoning from human videos.

  • Project
    HorizonRobotics/HoloAgent

    A unified embodied-agent framework for general-purpose robots that integrates 3D spatial memory and closed-loop execution for long-horizon real-world tasks.

  • Project
    Robbyant/lingbot-vla-v2

    LingBot‑VLA 2.0 is an open‑source Vision‑Language‑Action foundation model for robots. It unifies many robot embodiments into a 55‑dim action vector, uses MoE layers and dual‑query distillation (depth + future video) to learn predictive dynamics, and is trained on ~60 k h of robot and egocentric video data. The repo provides conda‑based installation, scripts to download 6 B‑parameter checkpoints, tools for data preparation, post‑training on custom datasets, and evaluation on the RoboTwin benchmark and real robots. Reported results show noticeable gains over the 1.0 version on bimanual and mobile manipulation tasks.