H-EmbodVis/TurboVLA

TurboVLA – Real‑time Vision‑Language‑Action model

What it is – TurboVLA is a research‑grade neural policy that maps raw camera images and natural‑language instructions directly to robot joint actions. It replaces the usual “vision → large language model → action” pipeline with a lightweight V + L → A architecture, achieving 32 Hz inference on an RTX 4090 while using < 1 GB VRAM.

Key ideas

• Separate encoders for vision (DINOv3 ViT‑B/L) and language (BERT‑base).
• A bidirectional vision‑language interaction module (built on GroundingDINO) lets the two modalities exchange information without a massive LLM.
• A compact decoder predicts continuous action chunks (12‑step for LIBERO, 50‑step for RoboTwin) instead of step‑by‑step commands, further cutting latency.

Performance

• LIBERO benchmark: 97.7 % average success, 0.2 B parameters, 31 ms latency, 0.9 GB VRAM.
• Matches or exceeds larger VLA baselines while being far cheaper to run.

How to get started

1. Clone & set up – The repo provides two optional environments:
   • turbovla-libero for the LIBERO simulation suite.
   • turbovla-robotwin for the RoboTwin 2.0 robot simulator.
2. Install dependencies – Use the provided pip install -e "[libero]" or "[robotwin]" extras after installing a CUDA‑compatible PyTorch build.
3. Download model & data – Model checkpoints and normalization stats are hosted on Hugging Face. Datasets (LIBERO TFDS/RLDS suites or RoboTwin Clean) are fetched via the helper scripts.
4. Train – Example torchrun commands show the exact hyper‑parameters used in the paper (batch size, optimizer steps, warm‑up, etc.).
5. Evaluate – Single‑command scripts evaluate a checkpoint on any LIBERO suite or on all 50 RoboTwin tasks.

Supported hardware – Primarily tested on consumer‑grade RTX 4090 GPUs. A TODO entry mentions future support for Huawei Ascend NPUs.

License – Apache 2.0.

Citation – The README provides a ready‑to‑copy BibTeX entry for the arXiv paper (arXiv:2607.27205).

---

Bottom line – TurboVLA is a concrete, open‑source implementation of a fast, low‑memory vision‑language‑action model for robotic manipulation, complete with training/evaluation scripts, environment setup instructions, and pretrained checkpoints.