toshas/torch-fidelity

What it solves

Generative models (like GANs) need to be measured objectively, but the most popular quality metrics are often reimplemented in ways that give slightly different numbers, run slowly, or are hard to reuse. torch-fidelity solves that by providing one fast, reliable toolbox for computing these metrics in PyTorch, with results that match the original reference implementations almost exactly.

How it works

It ships ready-to-use implementations of six well-known metrics: Inception Score (ISC), Fréchet Inception Distance (FID), Kernel Inception Distance (KID), Precision and Recall (PRC), Perceptual Path Length (PPL), and Monge Inception Distance (MIND). The library gets its speed from two tricks: it shares the computation of common features between different metrics, and it caches features and statistics so you don’t recompute them when you don’t have to. It is also designed to be modular — you can plug in your own feature extractor, which means the same metric logic can be applied to video, audio, 3D volumes, or anything else that has a learned representation.

Who it’s for

Researchers and engineers training or evaluating generative models in PyTorch. If you need to report standard quantitative metrics in a paper, track model quality during training, or compare different generative models fairly, torch-fidelity gives you a drop-in tool that is both precise and fast enough to run inside a training loop.

Highlights

• Numerically matches reference implementations to machine precision, so paper metrics are trustworthy.
• Fast: feature sharing between metrics and caching avoid recomputation, making it practical to evaluate at the end of every epoch.
• Extensible: register custom feature extractors to evaluate modalities beyond 2D images.
• Comes with both a command-line tool (fidelity) and a Python API (e.g. calculate_metrics).
• Can take many input types: built-in datasets like CIFAR-10, folders of images, or an ONNX generative model.