sii-research/tau-0-vla

This repo is the official implementation of "τ0-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation".

What it solves

τ₀-VLA is designed for long-horizon robot manipulation tasks. It addresses the challenge of executing complex sequences of actions across different robot embodiments by combining high-level reasoning and low-level execution.

How it works

The model uses a hierarchical architecture:

  1. High-Level Policy: A memory-augmented policy that determines the next subtask. It employs world-model-guided test-time computation to search for the best alternative actions when complex reasoning is required.
  2. Low-Level Policy: A generalist policy that executes the chosen subtask. This policy is built on a Qwen3.5 vision-language backbone and uses a Mixture-of-Transformers action expert trained via conditional flow matching. It operates on a unified 40-dimensional state/action space and was trained on over 40,000 hours of diverse real-world robot data.

Who it’s for

This project is for robotics researchers and developers working on foundation models for robot control, long-horizon manipulation, and generalist robot policies.

Highlights

  • Hierarchical Structure: Separates high-level subtask planning from low-level execution.
  • World-Model Guidance: Uses test-time computation to refine subtask selection through search.
  • Coflow Matching: Employs conditional flow matching for the action expert.
  • Cross-Embodiment: Trained on heterogeneous real-world data to work across different robot types.

Related

  • Project
    Robbyant/lingbot-vla

    A pragmatic Vision-Language-Action (VLA) foundation model pre-trained on 20,000 hours of real-world robot data to enable high-performance robotic control across multiple platforms.

  • Project
    Robbyant/lingbot-vla-v2

    LingBot-VLA 2.0 is a Vision-Language-Action foundation model that enables robots to generalize across different embodiments and tasks using a unified action representation and MoE action experts.

  • Project
    X-Square-Robot/wall-x

    An open-source framework for embodied foundation models that enables robots to perform complex physical manipulations by bridging vision-language understanding with action prediction.

  • Project
    open-gigaai/giga-brain-0

    A world model-powered Vision-Language-Action (VLA) foundation model that uses synthetic data to reduce reliance on real-world robot data for generalist robotics.

  • Project
    NVIDIA/Isaac-GR00T

    An open vision-language-action (VLA) model for generalized humanoid robot skills, enabling cross-embodiment manipulation tasks via multimodal input.