roboflow/rf-detr

What is RF‑DETR?

RF‑DETR (Real‑Time DETR) is an open‑source computer‑vision model family from Roboflow that uses a transformer‑based backbone (the DINOv2 vision transformer) to perform object detection, instance segmentation, and—currently in preview—keypoint detection. It is positioned as a real‑time alternative to classic DETR‑style models, delivering a strong accuracy‑latency trade‑off on standard benchmarks such as Microsoft COCO and the Roboflow‑specific RF100‑VL dataset.

---

Key Features (as described in the README)

Feature	Details
Unified API	One Python package (rfdetr) lets you load a detection, segmentation, or keypoint model with a single class name (e.g., RFDETRMedium).
Multiple Sizes	Six model families are shipped: Nano, Small, Medium, Large (Apache‑2.0 licensed) and XL / 2XL (available via the rfdetr_plus extension under the PML 1.0 license).
Backbone	All models share a DINOv2 vision‑transformer backbone, which is a state‑of‑the‑art frozen encoder for visual features.
Real‑time Performance	Latency is measured on an NVIDIA T4 (TensorRT, FP16, batch‑size 1). The smallest model runs at ~2 ms per image, while the largest stays under 20 ms, making the family suitable for edge or low‑latency deployments.
Benchmarks	The README includes extensive tables comparing COCO AP‑50, AP‑50:95, RF100‑VL scores, latency, parameter count and resolution against YOLO 11/26, LW‑DETR, D‑FINE, and even SAM 3. RF‑DETR consistently outperforms the listed competitors at comparable latency.
NAS Support	The architecture was discovered with neural‑architecture‑search (NAS). The same NAS pipeline is exposed on the Roboflow platform, allowing users to search for a custom‑tailored model for their own dataset.
Installation	pip install rfdetr (Python ≥ 3.10). Development builds are installable via a direct GitHub URL. The “plus” models require pip install rfdetr[plus].
Demo & Docs	Links to a Hugging Face Space, a Colab notebook, a blog post, and a Discord community are provided for quick hands‑on testing.
Licensing	Apache 2.0 for the core models; the larger XL/2XL models are under the Roboflow‑specific PML 1.0 license.

---

How to Use (quick example from the README)

import supervision as sv
from rfdetr import RFDETRMedium
from rfdetr.assets.coco_classes import COCO_CLASSES

model = RFDETRMedium()
# Predict on an image URL
preds = model.predict("https://media.roboflow.com/dog.jpg", threshold=0.5)

# Human‑readable labels (COCO pretrained)
labels = [COCO_CLASSES[i] for i in preds.class_id]

# Visualise
img = preds.metadata["source_image"]
img = sv.BoxAnnotator().annotate(img, preds)
img = sv.LabelAnnotator().annotate(img, preds, labels)

The same pattern works for the segmentation (RFDETRSegMedium) and keypoint models (preview). An alternative inference‑library interface (inference.get_model("rfdetr-medium")) is also shown.

---

Who Might Want This?

• Developers needing a drop‑in transformer‑based detector/segmenter that runs in real‑time on modest GPUs.
• Researchers interested in comparing transformer DETR‑style models against YOLO families on COCO‑style metrics.
• Roboflow platform users who want to leverage the built‑in NAS to auto‑tune the architecture for a custom dataset.

---

Where to Find More

• Paper: arXiv 2511.09554 (linked in the badge).
• GitHub: the repository you’re reading (contains the rfdetr package, model checkpoints, and benchmark scripts).
• Hugging Face Space and Colab notebook for interactive demos.
• Roboflow blog post and Discord for community support.

---

All statements above are taken directly from the repository’s README; no additional features have been inferred.