Qwen-RobotNav: A Scalable Navigation Model for Agentic Systems

TL;DR

Qwen-RobotNav is a scalable navigation model built on the Qwen3-VL backbone that unifies five distinct navigation tasks under a single set of weights. It introduces a controllable observation protocol that allows an agent to tune visual context at inference time, enabling state-of-the-art performance in vision-language navigation, object search, tracking, autonomous driving, and embodied question answering.

Unified Multi-Domain Navigation

Qwen-RobotNav achieves state-of-the-art (SOTA) results across five navigation domains using a single model with one set of weights. Performance scales consistently from 2B to 8B parameters. Key benchmarks include:

  • Vision-Language Navigation (VLN-CE): The 8B model achieves a 76.5% Success Rate (SR) on the RxR benchmark and 72.1% SR on R2R.
  • Object-Goal Navigation (HM3Dv2): The 4B model achieves a 75.6% SR using only RGB observations, surpassing depth-based methods.
  • Active Visual Tracking (EVT-Bench): The 4B model reaches a 90.0% tracking rate.
  • Autonomous Driving (NAVSIM): The 4B model achieves 91.4 PDMS, outperforming specialized driving models.
  • Embodied Question Answering (EQA): The agentic system sets new SOTA results across three benchmarks (HM-EQA, MT-EQA, and EXPRESS).

Controllable Observation Protocol

Qwen-RobotNav treats navigation context as a first-class, externally controllable interface. This approach allows the model to adapt to different task requirements—such as long-term memory for instruction following or high recency for target tracking—without architectural changes or retraining.

The model exposes four control axes as inference-time parameters:

  1. Visual token budget: Total tokens allocated across all cameras and timesteps.
  2. Temporal decay: The weight given to recent frames versus older ones.
  3. Camera weights: The relative importance of specific cameras (e.g., prioritizing the forward camera).
  4. Frame sample mode: Selection between random sampling for global history or latest-frame sampling for tight recency.

At training time, these parameters are randomized per sample, ensuring the model generalizes to any configuration during inference.

Agentic Navigation System Architecture

Qwen-RobotNav is designed as a reconfigurable primitive within a two-tier agentic system:

  • Upper-Level Planner: Powered by Qwen3.7-Plus, this tier decomposes long-horizon goals into sub-goals and dispatches configurable navigation calls. It dynamically switches task modes and context strategies mid-episode.
  • Navigation Primitive: Qwen-RobotNav executes these segments as a reactive waypoint predictor, outputting 8 waypoints (each with position and heading) via a 4-layer MLP action head.

To maintain long-term reasoning, the system employs a two-level memory: a compact trajectory summary for each segment and a persistent "evidence notebook" that tracks searched regions and rejected hypotheses.

This architecture resulted in a 15.4% improvement on EXPRESS-Bench with 77% fewer navigation steps compared to previous bests.

Training and Data Pipeline

The model was trained on 15.6 million samples across five task families, supplemented by vision-language reasoning data. To expand the dataset, Qwen introduced an automated pipeline that converts text-to-video generations into navigation trajectories. This process involves prompt generation, video synthesis, VLM quality filtering, monocular depth estimation, and kinematic filtering, adding 40K photorealistic samples without requiring 3D scene reconstruction.

Real-World Deployment and Generalization

Qwen-RobotNav was deployed zero-shot on a Unitree Go2 quadruped robot using an NVIDIA Jetson Thor for on-device inference, achieving a latency of 196ms (5.1 Hz).

Using only the robot's built-in low-resolution camera, the model demonstrated strong generalization in unseen environments:

  • Instruction Following: The robot successfully navigated 21.78m in an exhibition hall and precisely retraced the route in reverse upon command.
  • Agentic Navigation: The system autonomously handled open-ended requests, such as checking for a green umbrella at a specific coffee shop, by decomposing the task, localizing via landmarks, and providing an evidence-grounded answer.

Sources