Hugging Face Object Detection Leaderboard

Hugging Face released an Object Detection Leaderboard that ranks open-source models using COCO-style metrics and published a blog explaining how Average Precision and Average Recall are computed and what factors can influence the results.

Introduction

The blog explains the evaluation process for object detection models, detailing the metrics used on the leaderboard and the pitfalls that can cause divergent results.

What is Object Detection

Object detection identifies and localizes individual objects within an image, providing class labels, bounding boxes, and confidence scores for each detection.

Metrics

Average Precision (AP) summarizes the Precision‑Recall curve, while Average Recall (AR) focuses on recall across IoU thresholds.

What's Average Precision and how to compute it?

AP is the area under the Precision‑Recall curve for a class; Precision = TP/(TP+FP) and Recall = TP/(TP+FN), where TP/FP/FN depend on an IoU threshold.

What's Average Recall and how to compute it?

AR is the mean of the maximum recall obtained over IoU thresholds from 0.5 to 0.95, optionally limited by detection count or object size.

What are the variants of Average Precision and Average Recall?

Variants include AP@.5, AP@.75, AP@[.5:.05:.95] (mAP), AP‑S/M/L for small/medium/large objects, and AR‑1/AR‑10/AR‑100 plus AR‑S/M/L for recall‑focused evaluation.

Object Detection Leaderboard

The leaderboard ranks models using 12 COCO‑style metrics on COCO val 2017, with accuracy measured via PyCOCOtools and efficiency measured as frames per second with batch size = 1.

How to pick the best model based on the metrics?

Choose AP for overall performance, AP@.5 for lenient localization, AP@.75 for tight localization, AP‑S/M/L for size‑specific tasks, and AR variants when recall is the priority.

Which parameters can impact the Average Precision results?

Factors that can change AP include confidence‑score thresholds, batch size > 1, ported models with altered logits, ignored ground‑truth tags like iscrowd, IoU calculation details, and prompt quality for text‑conditioned models.

Conclusions

Evaluating object detection requires careful consideration of model specifics, dataset traits, and metric choice; the leaderboard provides a standardized comparison using community‑accepted COCO evaluation code.

Additional Resources

• Object Detection Guide
• Task of Object Detection
• Paper: What Makes for Effective Detection Proposals
• Paper: A Comparative Analysis of Object Detection Metrics with a Companion Open‑Source Toolkit
• Paper: A Survey on Performance Metrics for Object‑Detection Algorithms