starVLA/starVLA

StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing

⭐ What is StarVLA?

StarVLA is an open‑source research platform for building Vision‑Language‑Action (VLA) models – neural networks that can see, read language, and generate robot actions. The codebase is deliberately modular (“Lego‑like”), letting researchers swap components such as the visual‑language backbone, the action‑generation head, or the training recipe with minimal friction.

🎯 Core Capabilities

Area What StarVLA Provides
Modular Architecture Separate, high‑cohesion modules for data loading, model backbones (e.g., Qwen‑VL, Florence‑2), action heads (FAST, OFT, flow‑matching, GR00T), and training utilities. Each piece can be run in isolation for quick debugging.
Multi‑Benchmark Support Ready‑to‑run pipelines for a growing list of embodied‑AI benchmarks: LIBERO, LIBERO‑plus, SimplerEnv, RoboCasa‑GR1/365, RoboTwin 2.0, DOMINO, BEHAVIOR, CALVIN, RoboDojo, etc.
Training Recipes Implements supervised fine‑tuning, multimodal co‑training, cross‑embodiment co‑training, and a hook for reinforcement‑learning post‑training.
Backbone Flexibility Supports a range of vision‑language models: Qwen‑3 series (0.8 B‑9 B), Qwen‑3.5, Florence‑2, and the VLAct pre‑trained backbone. Also works on Ascend NPU and GPU clusters via DeepSpeed ZeRO.
Action Heads Four ready‑made heads:
  • FAST – discrete token actions.
  • OFT – parallel continuous MLP decoding.
  • PI – diffusion‑based flow‑matching.
  • GR00T – dual‑system (System 1 + System 2) architecture. | | Extensible Config System | All hyper‑parameters are stored in a single hierarchical YAML/Dict config, allowing easy overrides (e.g., swapping the VLM with --framework.qwenvl.base_vlm Qwen/Qwen2.5-VL-7B-Instruct). | | Community & Resources | Project page, model zoo on HuggingFace, Docker environments from AllenAI, and a contributors group for coordinated development. |

🚀 Getting Started (Quick‑Start)

  1. Install – Follow the step‑by‑step guide in docs/starVLA_guideline.md (environment setup, dependencies, optional Docker).
  2. Pick a Benchmark – Example: examples/simBenchmarks/LIBERO/train_files/starvla_cotrain_oxe.yaml.
  3. Run a Smoke Test (optional):
    python starVLA/model/framework/VLM4A/QwenOFT.py --config_yaml starvla_cotrain_oxe.yaml
python starVLA/dataloader/lerobot_datasets.py --config_yaml starvla_cotrain_oxe.yaml
  4. Launch Training – Using Accelerate/DeepSpeed:
    accelerate launch \
  --config_file starVLA/config/deepseeds/deepspeed_zero2.yaml \
  --num_processes 8 \
  starVLA/training/train_starvla.py \
  --config_yaml examples/simBenchmarks/LIBERO/train_files/starvla_cotrain_oxe.yaml \
  --framework.qwenvl.base_vlm Qwen/Qwen2.5-VL-7B-Instruct
  5. Evaluate – Run the provided evaluation harness (e.g., examples/simBenchmarks/RoboDojo/eval.sh). Results are automatically logged and can be compared against the published benchmark tables.

📊 Reported Performance

StarVLA claims state‑of‑the‑art results on several suites, e.g.:

  • LIBERO – see the bar‑chart in assets/starvla_LIBERO.png.
  • RoboCasa‑PI – 40.64 % success on a 1 016/2 500 task split.
  • SimplerEnv, Calvin, RoboTwin 2.0, DOMINO, and others (full tables in the Overleaf link).

🤝 Community & Contributions

  • Model & Data Hub – HuggingFace collection StarVLA/* hosts pretrained checkpoints and datasets.
  • Contributor List – Live at starvla.github.io/contributors and visualized via the contrib.rocks badge.
  • Discussion Channels – WeChat group (link in README) and GitHub issues for feature requests, bug reports, and PR reviews.
  • Derived Projects – NeuroVLA, PhysBrain, TwinBrainVLA, LangForce, LaWAM, etc., all build on StarVLA’s core stack.

📚 When to Use StarVLA?

  • Research on embodied AI where you need a unified pipeline for vision‑language perception and robot control.
  • Prototyping new action heads or experimenting with different VLM backbones without rewriting data pipelines.
  • Benchmarking across many simulated/real robot suites with a single codebase.
  • Scaling experiments using DeepSpeed, Ascend NPU, or multi‑GPU clusters.

📌 Citation

If you use StarVLA, cite the arXiv technical report (2604.05014) – the README provides an up‑to‑date BibTeX entry.

Related

  • Project
    Robbyant/lingbot-vla-v2

    LingBot‑VLA 2.0 is an open‑source Vision‑Language‑Action foundation model for robots. It unifies many robot embodiments into a 55‑dim action vector, uses MoE layers and dual‑query distillation (depth + future video) to learn predictive dynamics, and is trained on ~60 k h of robot and egocentric video data. The repo provides conda‑based installation, scripts to download 6 B‑parameter checkpoints, tools for data preparation, post‑training on custom datasets, and evaluation on the RoboTwin benchmark and real robots. Reported results show noticeable gains over the 1.0 version on bimanual and mobile manipulation tasks.

  • Project
    X-Square-Robot/wall-x

    An open-source framework for embodied foundation models that enables robots to perform complex physical manipulations by bridging vision-language understanding with action prediction.

  • Project
    2toinf/X-VLA

    X-VLA is a soft-prompted Transformer-based Vision-Language-Action model designed for scalable cross-embodiment robotic control across diverse hardware platforms.

  • Project
    FluxVLA/FluxVLA

    FluxVLA Engine is an open‑source, end‑to‑end platform for training, evaluating, and deploying vision‑language‑action (VLA) models for robots. It offers unified configuration, modular model back‑ends, benchmark suites (LIBERO, RoboCasa, RoboDojo), and deployment tools (Docker, Jetson Orin, Triton, ZMQ) with real‑robot demos on Franka and Oli.

  • Project
    Robbyant/lingbot-vla

    A pragmatic Vision-Language-Action (VLA) foundation model pre-trained on 20,000 hours of real-world robot data to enable high-performance robotic control across multiple platforms.