rafaelpadilla/Object-Detection-Metrics

What it solves

This project addresses the lack of consensus and the difficulty of implementing consistent evaluation metrics for object detection. Researchers often have to implement their own metrics when using datasets outside of major competitions, which can lead to biased or incorrect results. This toolkit provides a flexible, standardized implementation of popular metrics to ensure trustworthy benchmarking across different approaches and datasets.

How it works

The toolkit provides easy-to-use functions that implement metrics used by major competitions (such as PASCAL VOC and COCO) without requiring users to convert their detection models' output into complex XML or JSON formats. It simplifies the input process for ground truth and detected bounding boxes and calculates key performance indicators including:

• Intersection Over Union (IOU): Measures the overlap between predicted and ground truth bounding boxes to determine if a detection is a True Positive (TP) or False Positive (FP).
• Precision and Recall: Calculates the ability of the model to identify only relevant objects and find all relevant cases, respectively.
• Precision x Recall Curve: Plots performance as the confidence threshold changes.
• Average Precision (AP): Computes the area under the Precision x Recall curve using either 11-point interpolation or all-point interpolation.

Who it’s for

It is designed for researchers and developers working on object detection algorithms who need a reliable way to evaluate their models' accuracy across various datasets.

Highlights

• Competition-Standard Metrics: Implements metrics used by PASCAL VOC, COCO, and Google Open Images.
• Simplified Input: Avoids the need for complicated input format conversions (like XML or JSON).
• Verified Accuracy: Implementation has been compared against official competition codes to ensure identical results.
• Flexible Interpolation: Supports both the 11-point and all-point interpolation methods for Average Precision.