allenai/molmoact2

Official Repository for MolmoAct2

What it solves

MolmoAct2 provides a family of action reasoning models designed for robot control and real-world deployment. It addresses the challenge of connecting high-level visual-language reasoning (VLM) to continuous, closed-loop robot manipulation, allowing robots to execute tasks based on visual observations and language instructions.

How it works

The system builds upon the Molmo2-ER embodied-reasoning vision-language backbone. It integrates robot state and action modeling by connecting the VLM to a continuous flow-matching action expert. This architecture allows the model to reason about the environment and then translate that reasoning into precise robot actions.

Who it’s for

This project is intended for robotics researchers and developers working with robot embodiments such as the SO-100/101, Bimanual YAM, and Franka DROID setups, as well as those looking to fine-tune vision-language-action (VLA) models for specific manipulation tasks.

Highlights

  • Diverse Model Family: Includes base checkpoints for continued training, a "Think" version with depth-token reasoning, and fine-tuned policies for specific platforms (DROID, YAM, SO-100/101, LIBERO).
  • LeRobot Integration: Fully integrated into the Hugging Face LeRobot library for standardized training, evaluation, and deployment workflows.
  • Real-world Ready: Provides FastAPI inference servers for low-latency deployment and validated support for Intel XPU (GPUs).
  • Extensive Datasets: Releases the MolmoAct2-BimanualYAM, general robotics datasets, and Molmo2-ER embodied reasoning datasets.

Related

  • Project
    NVlabs/alpamayo1.5

    Alpamayo 1.5 is a Vision-Language-Action model for autonomous driving that combines chain-of-causation reasoning with a diffusion expert to predict driving trajectories.

  • Project
    dexmal/opendm

    OpenDM is a Vision-Language-Action (VLA) model framework for open-world robot control, providing tools for training, fine-tuning, and deploying models across various robot embodiments.

  • Dispatch
    SmolVLA: Efficient Vision-Language-Action Model trained on Lerobot Community Data

    Hugging Face introduces SmolVLA, a compact 450M parameter open-source Vision-Language-Action model that outperforms larger models on simulation and real-world robotics tasks using community-shared data.

  • Project
    NVlabs/alpamayo

    Alpamayo 1 is a Vision-Language-Action model for autonomous driving that combines reasoning traces with trajectory prediction to handle complex, rare driving scenarios.

  • Dispatch
    Hugging Face Vision Language Models 2025 Update

    Hugging Face provides a comprehensive overview of the 2024-2025 evolution of Vision Language Models (VLMs), highlighting trends in any-to-any architectures, reasoning models, and the rise of Vision-Language-Action (VLA) models for robotics.