ginwind/VLA-JEPA

[ECCV 2026] VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model

What it solves

VLA-JEPA improves Vision-Language-Action (VLA) models by integrating a latent world model. This allows the model to better understand and predict environment dynamics, enhancing the robot's ability to perform complex tasks across various benchmarks.

How it works

The project combines a Vision-Language Model (Qwen3-VL-2B) with a V-JEPA2 encoder to create a model capable of processing visual inputs and language instructions to output robotic actions. It supports training on both robot-specific datasets (in LeRobot v2.1 format) and human video datasets, utilizing a latent-action training approach with Chain-of-Thought (CoT) prompting.

Who it’s for

It is designed for robotics researchers and developers working on embodied AI, specifically those aiming to improve the generalizability and performance of VLA models in simulated and real-world environments.

Highlights

  • Supports training on diverse datasets including Droid, LIBERO, BridgeV2, and Fractal.
  • Compatible with LeRobot v2.1 data format for custom robot datasets.
  • Includes evaluation scripts for LIBERO, LIBERO-Plus, and SimplerEnv benchmarks.
  • Integrates Qwen3-VL and V-JEPA2 for enhanced visual and latent world modeling.

Related

  • Project
    InternRobotics/InternVLA-A-series

    InternVLA-A1.5 is a robotic policy that unifies vision-language understanding, visual foresight, and action generation to improve compositional generalization in robotic manipulation.

  • Project
    dexmal/opendm

    OpenDM is a Vision-Language-Action (VLA) model framework for open-world robot control, providing tools for training, fine-tuning, and deploying models that can handle complex, multi-embodiment tasks.

  • Project
    Robbyant/lingbot-vla

    A pragmatic Vision-Language-Action (VLA) foundation model pre-trained on 20,000 hours of real-world robot data to enable high-performance robotic control across multiple platforms.

  • Project
    X-Square-Robot/wall-x

    An open-source framework for embodied foundation models that enables robots to perform complex physical manipulations by bridging vision-language understanding with action prediction.

  • Project
    starVLA/starVLA

    An open-source research platform for developing Vision-Language-Action (VLA) models with a modular, plug-and-play codebase for generalist robotics.