OpenBMB/DeepThinkVLA

🤖 What is DeepThinkVLA?

DeepThinkVLA is a research‑grade Vision‑Language‑Action (VLA) model that learns to think before it acts. It builds on the public pi0‑FAST checkpoint and adds a hybrid decoder that first generates a chain‑of‑thought (CoT) reasoning trace and then, in a single forward pass, emits the robot’s action vectors. The authors train the model on the LIBERO simulated robot benchmark, first with supervised fine‑tuning (SFT) on a newly created embodied CoT dataset, and then with a short reinforcement‑learning (RL) stage that aligns the whole think‑act sequence to task success.

🎯 Core ideas

Idea	Why it matters
Hybrid attention decoder	Separates autoregressive reasoning from parallel action generation, keeping latency low while still allowing the model to reason about the scene.
Embodied CoT dataset	Two‑stage pipeline (cloud LVLM → fine‑tuned local VLM) produces high‑quality reasoning traces for whole robot trajectories.
Outcome‑driven RL (GRPO)	Uses sparse success rewards and a KL penalty to the SFT policy, giving a modest but consistent boost (+2 % SR) on long‑horizon tasks.
Mask‑CoT inference	Drops the reasoning tokens at inference time, preserving accuracy (96.5 % SR) while cutting runtime to 0.175× that of a pure autoregressive baseline.

📊 Reported performance

Metric	Value
Average success rate (SR) on LIBERO	97.0 %
Improvement over naive autoregressive CoT	+15.5 pp
RL boost on LIBERO‑Long	+2.0 pp
Inference latency (Mask‑CoT)	0.175× of pi0‑FAST
Zero‑shot SR on LIBERO Plus (trained only on LIBERO)	0.79  overall (see detailed breakdown in the README)

🛠️ Getting started

Environment – Linux/WSL, Python ≥ 3.10, CUDA 12.x. A typical SFT run needs ≥ 8 × 80 GB GPUs; the RL stage assumes a multi‑node setup.

Installation

conda create -n deepthinkvla python=3.10 -y
conda activate deepthinkvla
pip install -r requirements.txt
# optional fix for egl_probe on Windows
pip install cmake==3.31.6
wget https://github.com/mhandb/egl_probe/archive/fix_windows_build.zip
pip install fix_windows_build.zip

Data & checkpoints – All assets are hosted on Hugging Face. Example to pull the SFT checkpoint:

huggingface-cli download --repo-type model \
    --resume-download yinchenghust/deepthinkvla_libero_cot_sft \
    --local-dir ./checkpoints/sft/

Training – Supervised fine‑tuning:

bash scripts/finetune.sh   # wraps a deepspeed launch of src/train.py

Reinforcement‑learning refinement:

bash scripts/run_deepthinkvla_rl.sh

Evaluation – Use the provided eval script or the lightweight LIBERO Plus zero‑shot repo:
```
bash scripts/eval.sh --pretrained_checkpoint yinchenghust/deepthinkvla_libero_cot_rl
```

📂 Repository layout (high‑level)

data/ – helpers for downloading the CoT dataset and LIBERO simulation data.
scripts/ – launchers for SFT, RL, and evaluation.
src/
- configs/ – DeepSpeed and hyper‑parameter configs.
- dt_datasets/ – dataset wrappers, tokenizers, image normalisation.
- experiments/ – evaluation utilities and LIBERO runners.
- sft/ – model definition (hybrid decoder) and trainer.
- verl/ – VERL PPO implementation used in the RL stage.
figs/ – figures used in the README.

🧩 When would you use DeepThinkVLA?

Research on embodied reasoning – If you want a model that can generate explicit CoT traces for robot manipulation tasks.
Latency‑sensitive robot control – The hybrid decoder lets you keep inference fast while still benefiting from reasoning.
Benchmarking – Provides scripts and checkpoints for the LIBERO suite and a zero‑shot evaluation on the newer LIBERO Plus benchmark.
Dataset creation – The two‑stage CoT pipeline can be adapted to other robot‑vision datasets.

📚 Further reading & citations

Paper: DeepThinkVLA: Enhancing Reasoning Capability of Vision‑Language‑Action Models (arXiv:2511.15669, 2025).
Related repos: SimpleVLA‑RL, Qwen2‑VL‑Finetune, HybridFlow, LeRobot, openpi.
Citation (BibTeX provided in the README).

DeepThinkVLA is a full‑stack research codebase that demonstrates how adding a short, explicit reasoning phase can substantially improve success rates on simulated robot manipulation benchmarks while keeping inference efficient. It is ready for anyone with access to a multi‑GPU cluster to reproduce the results or to extend the ideas to new embodied AI tasks.