Thinklab-SJTU/Bench2DriveZoo

BEVFormer, UniAD, VAD in Closed-Loop CARLA Evaluation with World Model RL Expert Think2Drive

What it solves

This project provides a unified implementation and evaluation framework for several end-to-end autonomous driving models (BEVFormer, UniAD, and VAD) within the Bench2Drive benchmark. It specifically focuses on training these models as "students" using a world-model RL teacher called Think2Drive, while simplifying the complex dependency management typically associated with these architectures.

How it works

The repository integrates multiple fragmented dependencies (such as mmcv, mmseg, mmdet, and mmdet3d) into a single library to ensure compatibility with modern PyTorch versions and acceleration frameworks like DeepSpeed. It supports three primary stages of validation: training, open-loop evaluation, and closed-loop evaluation within the CARLA simulator. The models are trained to mimic the behavior of the Think2Drive teacher model to improve driving performance.

Who it’s for

This is designed for researchers and engineers working on end-to-end autonomous driving, specifically those looking to benchmark existing models like UniAD or VAD against the Bench2Drive standards or those interested in student-teacher learning paradigms for vehicle control.

Highlights

  • Unified Dependencies: Merges multiple MMLab libraries into one to reduce installation friction and support DeepSpeed.
  • Teacher-Student Framework: Implements models that learn from the Think2Drive world model RL teacher.
  • Comprehensive Evaluation: Includes code for both open-loop and closed-loop (CARLA) testing.
  • Pre-trained Models: Provides ready-to-use weights for UniAD-Tiny, UniAD-Base, VAD, and BEVFormer variants.

Related

  • Project
    Thinklab-SJTU/Bench2Drive

    A closed-loop dataset and benchmark for autonomous driving systems using the CARLA simulator to evaluate driving scores, success rates, and specific driving abilities.

  • Project
    Tencent-Hunyuan/UniRL

    UniRL is a reinforcement learning framework that provides a unified post-training loop for multimodal models, including image and video diffusion, autoregressive VLMs, and LLMs.

  • Project
    jolibrain/deepdetect

    A deep learning runtime and REST server that provides a unified API for training and inference across images, text, and tabular data.

  • Project
    ucla-mobility/OpenCDA

    An open-source co-simulation framework for Cooperative Driving Automation (CDA) that integrates CARLA and SUMO to research multi-vehicle cooperation and V2X communication.

  • Project
    dexmal/dexbotic

    A VLA (Vision-Language-Action) development toolbox for embodied intelligence that provides a unified framework for pretraining, fine-tuning, and evaluating robotic policies.