lartpang/PySODEvalToolkit

What it solves

It provides a standardized way to evaluate the performance of image segmentation models, specifically for grayscale and binary segmentation tasks. It eliminates the need for researchers to manually implement complex evaluation metrics or create custom plotting scripts for comparing different methods across multiple datasets.

How it works

The toolkit uses JSON configuration files to map dataset ground truths to model predictions based on shared filename patterns. It then calculates a wide array of metrics using the PySODMetrics library and can export these results into various formats. Users can run an evaluation script to generate metric data and a separate plotting script to visualize performance via PR, F-measure, and E-measure curves.

Who it’s for

Computer vision researchers and developers working on Saliency Object Detection (SOD), Camouflaged Object Detection (COD), and general binary image segmentation tasks.

Highlights

• Comprehensive Metrics: Supports a vast range of metrics for grayscale images (MAE, E-measure, S-measure, Weighted F-measure, etc.) and binary images (IoU, Dice, Precision, Recall).
• Flexible Exporting: Results can be exported to TXT, XLSX, or directly as LaTeX table code for academic papers.
• Automated Visualization: Includes built-in tools to plot professional-grade performance curves (PR, F-measure, E-measure) with customizable styles.
• Efficient Processing: Implements a concurrency strategy using multi-threading to speed up the computation of metrics across large datasets.