mll-lab-nu/VAGEN

World model reinforcement learning for multi-turn VLM agents. RL for vision framework (NeurIPS 2025).

What it solves

VAGEN addresses the challenge of training Vision-Language Model (VLM) agents to handle multi-turn tasks. It specifically aims to improve an agent's ability to reason about the world by reinforcing its internal "world model"—the ability to accurately estimate the current state and predict future transitions—rather than just optimizing for final task success.

How it works

The framework treats multi-turn VLM tasks as a Partially Observable Markov Decision Process (POMDP). It uses a "World Modeling RL" approach where an LLM-as-Judge provides rewards not only for achieving the goal but also for the accuracy of the agent's visual state reasoning (StateEstimation and TransitionModeling).

Technically, VAGEN provides a modular architecture that decouples the environment, the agent loop (harness), the model adaptation, and the reinforcement learning algorithm. This allows researchers to swap out different VLM backends (like Qwen-VL, InternVL, or GLM) and different advantage estimators (like GAE or GRPO) while maintaining a consistent training and evaluation pipeline.

Who it’s for

AI researchers and developers working on embodied AI, robotic control, and multi-turn visual agents who need a standardized framework to train and evaluate VLMs in complex environments.

Highlights

  • World Model Reinforcement: Explicitly supervises the reasoning process through state estimation and transition prediction rewards.
  • Modular Design: Separate layers for environments, harnesses, and rollout execution, making it easy to add custom environments or training algorithms.
  • Broad Model Support: Compatible with various VLM families including Qwen2.5-VL, Qwen3-VL, InternVL3.5, and GLM-4.6V.
  • Flexible Harnesses: Supports multiple conversation formats (concat, no_concat, compact) to control how history is passed to the model.

Related

  • Project
    Alibaba-NLP/VRAG

    A multi-turn multimodal RAG framework that uses agentic reinforcement learning to enable VLMs to reason and retrieve information across text, images, and videos.

  • Project
    langfengQ/verl-agent

    An extension of veRL for training LLM agents via reinforcement learning, featuring a step-independent rollout mechanism to efficiently handle long-horizon, multi-turn tasks.

  • Project
    mll-lab-nu/RAGEN

    A flexible RL framework for training reasoning LLM agents using multi-turn reinforcement learning and the StarPO algorithm to prevent reasoning collapse.

  • Project
    TsinghuaC3I/MARTI

    An open-source framework for training LLM-based multi-agent systems using reinforcement learning, featuring tree search augmentation for complex reasoning and code generation.

  • Project
    AgentR1/Agent-R1

    A modular framework for Agentic Reinforcement Learning that trains multi-step LLM agents by treating each interaction turn as a step-level MDP transition.