lyuwenyu/RT-DETR

[CVPR 2024] Official RT-DETR (RTDETR paddle pytorch), Real-Time DEtection TRansformer, DETRs Beat YOLOs on Real-time Object Detection. 🔥 🔥 🔥

What it solves

RT-DETR is designed to overcome the limitations of traditional real-time object detection models, aiming to outperform YOLO-style detectors in both speed and accuracy. It provides a high-performance alternative for identifying and locating objects in images in real-time, specifically targeting complex scenarios and difficult environmental conditions.

How it works

The project implements a Real-Time Detection Transformer (DETR) architecture. It offers multiple model sizes (from R18 to X) and supports both PyTorch and PaddlePaddle frameworks. The system is optimized for high throughput (FPS) and high mean Average Precision (mAP) on standard benchmarks like COCO and Objects365. It also supports deployment via ONNX Runtime, TensorRT, and OpenVINO.

Who it’s for

Computer vision engineers and researchers who need a real-time object detection system that balances high accuracy with low latency, particularly for deployment on edge devices or high-performance GPUs.

Highlights

  • High Efficiency: Achieves high frames-per-second (FPS) on T4 TensorRT FP16, with the smallest model (R18) reaching 217 FPS.
  • Multiple Implementations: Official support for both PyTorch and PaddlePaddle.
  • Broad Deployment Support: Compatible with ONNX Runtime, TensorRT, and OpenVINO.
  • Iterative Improvements: Includes RT-DETRv2, which introduces a "bag-of-freebies" to further improve baseline performance.
  • Scalable: Offers a range of model sizes to fit different hardware constraints.

Related

  • Project
    Peterande/D-FINE

    D‑FINE is a real‑time object‑detection framework that redefines DETR’s box regression as a fine‑grained distribution refinement and adds a self‑distillation module. It offers several model sizes (‑N to ‑X) that achieve state‑of‑the‑art COCO/AP scores (42.8 %–55.8 %) while running at 70‑470 FPS on a T4 GPU, with no extra inference cost. The repo provides pretrained checkpoints, config files, and full training/evaluation scripts for COCO, Objects365, and custom COCO‑style datasets.

  • Project
    ultralytics/yolov3

    Ultralytics YOLOv3 is a PyTorch implementation of the YOLOv3 object‑detection model, providing three variants (full, SPP, tiny) with scripts for training, inference, validation, and export to many deployment formats. It includes automatic COCO‑pretrained weights, supports PyTorch Hub loading, and integrates with popular ML‑ops tools and cloud notebooks. Licensed under AGPL‑3.0 with a commercial Enterprise option.

  • Project
    z1069614715/objectdetection_script

    A comprehensive toolkit for object detection that provides modified YOLO and RT-DETR architectures, model pruning and distillation scripts, and dataset processing tools for research and deployment.

  • Project
    roboflow/rf-detr

    RF‑DETR is an open‑source, transformer‑based library for real‑time object detection, instance segmentation, and keypoint detection. Built on a DINOv2 backbone, it offers a family of models (from tiny “Atto” to large “2XL”) that achieve state‑of‑the‑art COCO accuracy while running in 1–17 ms on an NVIDIA T4. Install via `pip install rfdetr`, use a single Python API for all three tasks, and optionally run NAS on the Roboflow platform to tailor a model to your own dataset.

  • Project
    ultralytics/yolov5

    YOLOv5 is a PyTorch‑based, open‑source object detection/segmentation/classification model suite from Ultralytics, offering multiple model sizes, simple Python/CLI inference, one‑line training, and export to many deployment formats.