Qwen-Drive-1.0 release notes / what's new

TL;DR

Qwen-Drive-1.0 is the first vision-language foundation model for autonomous driving that integrates 3D perception and visual question answering (VQA) during pretraining and extends to motion planning. It achieves this by adding external perception and planning modules to a pretrained VLM, allowing it to handle driving-specific tasks while maintaining general-purpose vision-language capabilities.

Model Architecture

Qwen-Drive-1.0 is built upon the natively multimodal Qwen3.5-4B. The model maintains the original pretrained VLM architecture entirely untouched, instead utilizing two external modules that read from a shared vision encoder and VLM pathway processing single-view, multi-view, and temporal image sequences.

BEV Perception Head

The BEV (Bird's-Eye View) perception head acts as an explicit, inspectable 3D probe. It constructs a BEV representation from multi-view single-frame inputs to jointly perform three primary tasks:

  • 3D object detection
  • Semantic occupancy prediction
  • BEV map segmentation

During joint training, the losses from this head provide an additional gradient path into the shared visual pathway.

Planning Expert

The Planning Expert is a diffusion transformer designed for VLM representations. It uses flow matching to generate 5-second ego trajectories. This module can optionally be conditioned on textual planning reasoning to guide the trajectory generation.

Training Methodology

To adapt the general-purpose VLM to autonomous driving without catastrophic forgetting, Qwen-Drive-1.0 employs a staged training and data recipe. This process includes:

  • Label Unification: Unifying cross-dataset labels.
  • Response Rewriting: Rewriting responses for consistency.
  • Sample Filtering: Filtering samples to ensure data quality.
  • Hybrid Supervision: Combining driving-specific data with general-purpose vision-language supervision.

Performance and Benchmarks

Driving Scene Understanding

Qwen-Drive-1.0-SFT demonstrates superior driving scene understanding, achieving a driving QA average of 69.43, outperforming both general-purpose VLMs and driving specialists. Key benchmark results include:

  • LingoQA: 77.80
  • Ego3D (RMSE): 7.78 (lower is better)
  • WaymoQA (All): 74.47
  • SURDS: 66.13

Notably, the model retains high performance in general knowledge, reasoning, and recognition tasks, such as scoring 85.53 on MMBench and 75.87 on MMStar.

Motion Planning

Qwen-Drive-1.0 was trained on 2.83 million samples of publicly available data with unified trajectory formats (5-second predictions at 10 Hz). It shows competitive results in open-loop and closed-loop evaluations:

  • WOD-E2E (RFS val/test): 8.45 / 7.91
  • WOD-E2E (ADE 5s val/test): 1.27 / 2.67
  • NAVSIM (PDMS): 90.7 (SFT version)
  • AlpaSim (at-fault score): 0.37

Future Work

While Qwen-Drive-1.0 establishes a unified base for driving-scenario adaptation, the Qwen team identifies the need to strengthen the consistency between the model's textual reasoning and its generated trajectories as a primary focus for future development.

Sources