roboflow/rf-detr

RF-DETR is a real-time object detection and segmentation model architecture developed by Roboflow, SOTA on COCO, designed for fine-tuning. [ICLR 2026]

What is RF‑DETR?

RF‑DETR (Real‑Time DEtection TRansformer) is an open‑source computer‑vision library that provides a family of transformer‑based models for three core tasks:

  • Object detection – locate and label objects in an image.
  • Instance segmentation – produce a pixel‑wise mask for each detected object.
  • Keypoint detection – predict body‑pose style points (currently a preview feature).

All of these capabilities are exposed through a single, consistent Python API. The models are built on top of the DINOv2 vision‑transformer backbone (the “PE‑Core‑T” family) and have been tuned with neural‑architecture‑search (NAS) to give a strong accuracy‑latency trade‑off for real‑time use.

Why it matters

  • State‑of‑the‑art performance – On the standard COCO benchmark the largest RF‑DETR‑L model reaches 75.1 % AP₅₀ and 56.5 % AP (average precision across IoU thresholds), while the smallest “Atto” variant runs in ~1 ms on an NVIDIA T4. This puts it in the same speed class as YOLO‑style detectors but with higher accuracy.
  • Unified API – One model class can be used for detection, segmentation, or (preview) pose estimation, simplifying code bases that need multiple vision tasks.
  • Scalable model zoo – Sizes range from Nano (30 M parameters) up to 2XL (≈127 M parameters), letting you pick a model that fits your hardware budget.
  • Open‑source & extensible – The core rfdetr package is released under Apache 2.0. Larger “Plus” models are available under a permissive PML 1.0 license.
  • NAS on the Roboflow platform – The same NAS pipeline used to create the paper models is exposed as a service, so you can automatically search for a custom architecture that best fits your own dataset.

How to get started (in a few lines of code)

import supervision as sv
from rfdetr import RFDETRMedium          # pick any size: Atto, Nano, Small, …
from rfdetr.assets.coco_classes import COCO_CLASSES

model = RFDETRMedium()                  # loads the pre‑trained COCO weights

# Run inference on an image URL
preds = model.predict("https://media.roboflow.com/dog.jpg", threshold=0.5)

# Human‑readable labels (COCO class names)
labels = [COCO_CLASSES[i] for i in preds.class_id]

# Visualise the results
annotated = sv.BoxAnnotator().annotate(preds.metadata["source_image"], preds)
annotated = sv.LabelAnnotator().annotate(annotated, preds, labels)

The same model can be used via the Inference library (get_model("rfdetr-medium")) if you prefer a higher‑level wrapper.

Benchmarks at a glance

Model COCO AP₅₀ COCO AP (50:95) Latency (ms, T4, FP16) Params (M) License
RF‑DETR‑Atto (preview) 49.4 30.5 1.0 7.4 PML 1.0
RF‑DETR‑Nano 67.6 48.4 2.3 30.5 Apache 2.0
RF‑DETR‑Medium 73.6 54.7 4.4 33.7 Apache 2.0
RF‑DETR‑Large 75.1 56.5 6.8 33.9 Apache 2.0
RF‑DETR‑2XL (preview) 78.5 60.1 17.2 126.9 PML 1.0

Latency numbers are measured on an NVIDIA T4 GPU using TensorRT (FP16, batch‑size 1).

Where to find more

TL;DR

RF‑DETR is a real‑time, transformer‑based vision library that delivers high‑accuracy detection, segmentation, and (preview) pose estimation across a range of model sizes. It’s packaged as a pip‑installable rfdetr library, comes with Apache‑licensed checkpoints, and includes tools for custom NAS‑driven model search on the Roboflow platform.

Related

  • Project
    Peterande/D-FINE

    D‑FINE is a real‑time object‑detection framework that redefines DETR’s box regression as a fine‑grained distribution refinement and adds a self‑distillation module. It offers several model sizes (‑N to ‑X) that achieve state‑of‑the‑art COCO/AP scores (42.8 %–55.8 %) while running at 70‑470 FPS on a T4 GPU, with no extra inference cost. The repo provides pretrained checkpoints, config files, and full training/evaluation scripts for COCO, Objects365, and custom COCO‑style datasets.

  • Project
    lyuwenyu/RT-DETR

    A real-time object detection framework based on the Detection Transformer (DETR) architecture that aims to outperform YOLO models in speed and accuracy.

  • Project
    roboflow/roboflow-python

    Roboflow‑python is the official Python client for the Roboflow computer‑vision platform. It lets you programmatically create workspaces/projects, upload and version datasets, trigger model training, and run inference on hosted or self‑hosted models. Install via `pip install roboflow` (or the lightweight `roboflow-slim`), authenticate with an API key or login, and use a simple object hierarchy (workspace → project → version) to manage the full CV lifecycle.

  • Project
    autodistill/autodistill

    autodistill is a Python library that automatically labels images with a large foundation model and then trains a small, fast detector (e.g., YOLOv8) on that auto‑labeled data, producing an edge‑ready model without any human annotation.

  • Project
    deepinsight/insightface

    InsightFace is an open‑source 2D/3D face analysis library offering detection, recognition, alignment, video redaction, person‑level matching, and a self‑hosted server with REST API. Implemented in PyTorch/MXNet, it provides pretrained models, automatic hardware backend selection, and tools for large‑scale face search.