NVlabs/alpamayo1.5

NVIDIA Alpamayo 1.5 Nano is an open 10B reasoning VLA model for autonomous vehicles with reinforcement-learning enhanced reasoning, navigation guidance, and visual question answering.

What it solves

Alpamayo 1.5 is designed to bridge the gap between high-level reasoning and concrete action prediction for autonomous driving, particularly in "long tail" (rare or complex) scenarios. It provides a foundation for developing end-to-end driving backbones or reasoning-based auto-labeling tools.

How it works

The system uses a Vision-Language-Action (VLA) architecture featuring a Cosmos-Reason backbone and a diffusion expert. The process follows a two-step pipeline:

  1. Reasoning: A Vision-Language Model (VLM) generates a "chain-of-causation" reasoning trace.
  2. Action: A diffusion expert produces trajectory predictions (6.4s horizon, 64 waypoints at 10 Hz) conditioned on the VLM's hidden states.

Alpamayo 1.5 has been post-trained using Reinforcement Learning (RL) to improve the alignment between its reasoning and the resulting trajectories.

Who it’s for

Researchers and developers in the autonomous vehicle (AV) domain who need a pre-trained reasoning model to build customized driving applications, conduct scientific inquiry, or perform benchmarking.

Highlights

  • Interactive and Steerable: Supports explicit navigation guidance and variable numbers of input cameras.
  • Multimodal Capabilities: Capable of both trajectory prediction and general Visual Question Answering (VQA).
  • RL Post-trained: Enhanced reasoning quality and trajectory alignment compared to the original Alpamayo 1.
  • Hardware Optimized: Includes optional CUDA graph acceleration to reduce diffusion-expert launch overhead.

Related

  • Project
    NVlabs/alpamayo

    Alpamayo 1 is a Vision-Language-Action model for autonomous driving that combines reasoning traces with trajectory prediction to handle complex, rare driving scenarios.

  • Project
    dexmal/opendm

    OpenDM is a Vision-Language-Action (VLA) model framework for open-world robot control, providing tools for training, fine-tuning, and deploying models across various robot embodiments.

  • Project
    OpenGalaxea/GalaxeaVLA

    GalaxeaVLA is a vision-language-action model for general-purpose robot control that integrates reasoning and action into a single autoregressive stream for complex manipulation tasks.

  • Project
    om-ai-lab/VLM-R1

    VLM-R1 is a framework for training R1-style Large Vision-Language Models using GRPO to improve reasoning and generalization in tasks like visual grounding, math, and object detection.

  • Project
    xiaomi-research/recogdrive

    A reinforced cognitive framework for end-to-end autonomous driving that combines a Vision-Language Model with a diffusion planner to generate safe, continuous trajectories.