lartpang/PyIRSTDMetrics

[NeurIPS 2025 (D&B)] Rethinking Evaluation of Infrared Small Target Detection

What it solves

PyIRSTDMetrics provides a standardized and efficient way to analyze the performance of Infrared Small Target Detection (IRSTD) models. It addresses the need for consistent evaluation metrics that can handle both pixel-level accuracy and target-level detection success, specifically for the challenges associated with detecting very small targets in infrared imagery.

How it works

The library implements a suite of evaluation metrics based on numpy, scikit-image, and scipy. It supports two primary analysis levels:

  • Pixel-level analysis: Calculates standard metrics like IoU (Intersection over Union), F1-score, Precision, Recall, TPR, and FPR.
  • Target-level analysis: Focuses on detection probability (Pd) and false alarm rate (Fa) using various matching strategies, including a custom-designed OPDCMatching for more reasonable target matching.
  • Hybrid analysis: Introduces hIoU, a new metric that balances pixel-level and target-level performance, along with specialized tools for analyzing localization and segmentation errors.

Who it’s for

This tool is designed for researchers and developers working on computer vision and infrared target detection, particularly those implementing and testing new IRSTD algorithms.

Highlights

  • Comprehensive Metrics: Supports a wide range of pixel-level, target-level, and hybrid metrics.
  • Error Analysis: Includes dedicated tools for hierarchical IoU-based localization and segmentation error analysis.
  • Lightweight: Built on standard scientific Python libraries for fast and easy integration.
  • Visualization Support: Provides data sequences that can be used to plot PR (Precision-Recall) and ROC (Receiver Operating Characteristic) curves.

Related

  • Project
    YuChuang1205/PAL

    A progressive active learning framework for infrared small target detection that enables models to achieve high performance using only single-point supervision instead of full masks.

  • Project
    rafaelpadilla/Object-Detection-Metrics

    A toolkit providing standardized, easy-to-use implementations of popular object detection metrics like Average Precision and IOU to ensure consistent benchmarking across different datasets.

  • Project
    Koldim2001/YOLO-Patch-Based-Inference

    A Python library that enables patch-based inference for YOLO models to improve the detection of small objects in large images through image tiling and result consolidation.

  • Project
    cheind/py-motmetrics

    A Python library for evaluating multiple object tracking results using standard metrics like MOTA, MOTP, IDF1, and HOTA.

  • Project
    lynote-ai/ai-image-detector

    AI Image Detector is an open‑source Python tool that estimates how likely an image was generated by AI. It ships with several ready‑made back‑ends (UnivFD, Sentry, Nonescape, hybrid ensembles) and lets you run a single command (`aidetect detect …`) on a file or folder, outputting a probability and label. Optional extras give a Gradio web UI or a FastAPI server. The repo includes benchmarking scripts, threshold‑calibration, and clear guidance on which backend to pick. It’s a lightweight, install‑once‑run‑anywhere utility for AI‑image detection, with documented limitations (no universal guarantee, no region‑level localization).