QwenLM/Qwen-Drive-1.0

What it solves

Qwen-Drive-1.0 is a vision-language foundation model designed for autonomous driving. It addresses the need for a unified framework that can handle 3D perception, visual question answering (VQA) about driving scenes, and motion planning (generating future ego trajectories) within a single model architecture.

How it works

The model is built upon the pretrained Qwen3.5-4B vision-language model (VLM), which serves as a shared representation engine. Two specialized external modules are attached to this shared VLM:

• BEV Perception Head: Performs 3D object detection, semantic occupancy prediction, and Bird's-Eye-View (BEV) map segmentation to provide an explicit interface to the 3D scene structure.
• Planning Expert: Generates future ego trajectories based on the shared VLM representations.
• LLM Decoder: The original decoder remains unchanged to handle both general and driving-specific VQA tasks.

The model uses a staged training strategy that combines driving-specific supervision with general-purpose vision-language data to maintain general visual understanding while gaining specialized driving competence.

Who it’s for

This project is for researchers and developers working on autonomous driving, embodied intelligence, and vision-language models (VLMs) applied to robotics and vehicle navigation.

Highlights

• Unified Framework: Integrates perception, language, and planning in one model.
• Multimodal Capabilities: Handles 3D perception, driving-specific VQA, and motion planning.
• Staged Training: Balances specialized driving skills with general-purpose instruction-following.
• Flexible Inference: Supports different planning modes, including imitation-trained (SFT) and reward-optimized (RL) experts.
• High Performance: Outperforms several other models on driving-scene understanding and spatial grounding benchmarks.