sou350121/VLA-Handbook

本项目旨在为致力于进入VLA(Vision-Language-Action)领域的算法工程师提供一份全中文、实战导向的学习/面试手册。 不同于通用的 CV/NLP 面试指南，本项目聚焦于 Robotics 特有的挑战

VLA Handbook – A Living Knowledge Base for Vision‑Language‑Action Robotics

What it is – An open‑source, continuously‑updated repository that curates and distills the rapidly‑growing literature on Vision‑Language‑Action (VLA) models for robotics. It bridges the gap between reading a paper and actually running the code on a robot, providing:

  • Theory – 688 markdown analyses covering architecture, action representations, world models, tactile perception, etc.
  • Deployment – Practical guides, hardware recommendations, and “sanity‑check” scripts for reproducing results on real robots.
  • Community insights – Automated extraction of Chinese (Xiaohongshu) and English (HF blog, Discord) community tips, plus distilled GitHub Issues.
  • Industry radar – Daily tracking of robotics/embodied‑AI companies, financing, IPOs, and a weekly “industry judgement map”.
  • AI‑assistant skill – A plug‑in that injects the handbook’s knowledge into Claude, Cursor, Codex, etc.

Why it matters – VLA research is exploding (dozens of papers per week) but reproducibility is fragmented across GitHub issues, appendix code, and scattered forum posts. This project automates the collection, rating, and summarisation of new papers, then adds engineering‑level details (hyper‑parameters, shape checks, entry scripts) so developers can go from read to run in a single step.

Key components

Component What you get Update cadence
theory/ 688 deep‑dive markdowns (core, diffusion‑flow, world‑model, tactile, etc.) 2‑3 new analyses daily
deployment/ Hardware selection, multimodal sync, Sim‑2‑Real checklists, community field‑notes Chinese notes every 3 days, English notes weekly
reports/weekly & biweekly Trend digests, SOTA tables, prediction vs. reality scores Weekly / bi‑weekly
scripts/ Full pipeline (paper scraping, rating, summarising, RSS generation) – documented in SCRIPTS.md 33 cron jobs run each morning
companies/ Company profiles, financing, product updates, job‑hunt guides Daily radar, weekly map
VLA‑expert‑skill Ready‑to‑load skill for LLM coding assistants Syncs with handbook daily

How it works – The Pulsar engine (a separate repo) runs 33 cron jobs that:

  1. Pull the latest VLA papers from arXiv and other sources.
  2. Rate each paper with ⚡/🔧/📖/❌ tags (impact, engineering readiness, etc.).
  3. Generate a markdown analysis, update the index, and publish RSS feeds.
  4. Harvest community posts (Xiaohongshu, HF blog, Discord) and GitHub Issues, distill them into “field notes”, and merge them back.
  5. Maintain a set of 19 domain hypotheses, adjusting confidence scores automatically based on observed data – a self‑evolving knowledge‑validation loop.

Who should use it

  • Robotics engineers who need concrete entry scripts, hyper‑parameter tables, and hardware compatibility matrices.
  • Researchers looking for a curated, up‑to‑date map of VLA methods and trends.
  • Product teams tracking the competitive landscape of embodied‑AI startups.
  • LLM‑powered coding assistants that can be turned into VLA experts via the provided skill.

Getting started

  1. Clone the repo.
  2. Browse theory/README.md for a learning roadmap or jump to a specific topic (e.g., theory/vla-core/vla_arch.md).
  3. For real‑robot deployment, read deployment/README.md and follow the provided sanity‑check scripts.
  4. Subscribe to the RSS feeds (or import the OPML) to receive daily updates directly in your reader.
  5. Install the VLA Expert Skill into your preferred AI coding assistant to query the handbook conversationally.

Contribution – The project welcomes community contributions: new paper analyses, additional deployment experiences, interview question banks, or updates to the industry radar. See CONTRIBUTING.md for guidelines.

All information above is taken directly from the repository’s README; no external assumptions have been added.

Related

  • Project
    starVLA/starVLA

    StarVLA is a modular, open-source codebase for developing Vision-Language-Action models for robots. It offers plug-and-play components, multiple VLA frameworks, training recipes, and benchmark integrations, with a focus on rapid prototyping and easy customization.

  • Project
    allenai/vla-evaluation-harness

    vla‑evaluation‑harness is a Python library that lets you evaluate Vision‑Language‑Action models on dozens of robot‑simulation benchmarks (LIBERO, CALVIN, SimplerEnv, etc.) using Docker‑isolated environments and lightweight model‑server scripts. It provides batch‑parallel execution (sharding episodes + GPU batching) for huge speed‑ups, a unified leaderboard of 2 456 models, and a simple CLI/Python API for both research and training‑time evaluation.

  • Project
    Robbyant/lingbot-vla-v2

    LingBot‑VLA 2.0 is an open‑source Vision‑Language‑Action foundation model for robots. It unifies many robot embodiments into a 55‑dim action vector, uses MoE layers and dual‑query distillation (depth + future video) to learn predictive dynamics, and is trained on ~60 k h of robot and egocentric video data. The repo provides conda‑based installation, scripts to download 6 B‑parameter checkpoints, tools for data preparation, post‑training on custom datasets, and evaluation on the RoboTwin benchmark and real robots. Reported results show noticeable gains over the 1.0 version on bimanual and mobile manipulation tasks.

  • Project
    FluxVLA/FluxVLA

    FluxVLA Engine is an open‑source, end‑to‑end platform for training, evaluating, and deploying vision‑language‑action (VLA) models for robots. It offers unified configuration, modular model back‑ends, benchmark suites (LIBERO, RoboCasa, RoboDojo), and deployment tools (Docker, Jetson Orin, Triton, ZMQ) with real‑robot demos on Franka and Oli.

  • Project
    dsta022/Loop-Engineering-for-VLA

    Loop Engineering for VLAnything is an end‑to‑end Python toolkit that lets you record, merge, audit, enrich, and iteratively improve multimodal robot‑learning datasets (RGB, RGB‑D, and future visuo‑tactile) for vision‑language‑action models. It provides a conservative, side‑car‑first quality audit, plug‑in semantic evaluators, language‑annotation pipelines, and a closed‑loop feedback loop for policy improvement, all compatible with the LeRobot data format and publishable to Hugging Face.