Peterande/D-FINE

D‑FINE – Real‑time Object Detection with Fine‑grained Distribution Refinement

What it is – D‑FINE is a family of object‑detectors built on the DETR (DEtection TRansformer) architecture. The authors reinterpret the usual bounding‑box regression as a Fine‑grained Distribution Refinement (FDR) problem and add a Global Optimal Localization Self‑Distillation (GO‑LSD) module. The result is a set of models (‑N, ‑S, ‑M, ‑L, ‑X) that run at real‑time speeds (hundreds of FPS on an NVIDIA T4) while achieving state‑of‑the‑art COCO/AP scores.

Key ideas

• FDR: instead of predicting a single box coordinate, the network predicts a discrete distribution over possible locations and refines it iteratively, giving more precise localization without extra compute.
• GO‑LSD: a self‑distillation scheme that teaches the model to produce globally optimal box predictions, again without extra inference cost.
• No extra overhead: both tricks are baked into the training pipeline; inference uses the same lightweight backbone (HGNetV2 variants) and transformer decoder as vanilla DETR.

Model zoo – Pre‑trained checkpoints are provided for:

• COCO (standard detection benchmark) – five sizes from ~4 M to 62 M parameters, AP ranging from 42.8 % (‑N) to 55.8 % (‑X).
• Objects365 + COCO – models first pre‑trained on the larger Objects365 dataset and then fine‑tuned on COCO, yielding higher AP (up to 59.3 %).
• Objects365 only – for users who want a strong generic detector to fine‑tune on their own data.

All checkpoints come with config files (YAML) and training logs.

Getting started

1. Environment – create a conda env with Python 3.11, install the requirements.txt.
2. Data – the repo supports COCO‑2017, Objects365, CrowdHuman, or any custom dataset formatted like COCO. Detailed path‑setting instructions are in the README.
3. Training – pick a model size via export model=l (n/s/m/l/x) and launch training with torchrun across GPUs, e.g.

   CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 \
       --nproc_per_node=4 train.py -c configs/dfine/dfine_hgnetv2_${model}_coco.yml \
       --use-amp --seed=0
   

4. Testing / fine‑tuning – similar commands with --test-only or -t <ckpt>.

Performance

• Latency is measured on a single T4 GPU (fp16, TensorRT 10.4) with batch‑size 1. The smallest model (‑N) runs in ~2 ms (≈ 472 FPS) while the largest (‑X) runs in ~13 ms (≈ 77 FPS).
• FLOPs range from 7 GFLOPs (‑N) to 202 GFLOPs (‑X).
• The repo includes a video demo where D‑FINE outperforms YOLO‑11 on a challenging street‑scene clip.

Licensing & notes

• Code is under Apache 2.0. Checkpoints trained on Objects365 inherit that dataset’s license restrictions; they are not automatically cleared for commercial use.
• The project is actively maintained (updates listed up to 2024‑11‑07) and provides a Hugging Face Space for interactive inference.

Who should use it

• Researchers needing a fast, high‑accuracy transformer‑based detector.
• Practitioners who want a ready‑to‑run model for real‑time applications (e.g., video analytics, robotics) without the heavy compute of larger DETR variants.
• Anyone looking to fine‑tune on a custom COCO‑style dataset; the repo supplies scripts for data preparation and class‑mapping adjustments.

---

All details above are taken directly from the repository’s README; no additional features are inferred.