frgfm/torch-cam

What it solves

TorchCAM provides a simple way to visualize and explore class-specific activations of convolutional layers in PyTorch models. It helps developers debug surprising classifier results by generating heatmaps that highlight which parts of an image the model is focusing on to make a specific prediction.

How it works

The library uses PyTorch hooking mechanisms to retrieve internal model information without requiring the user to modify the model's architecture. A CAM extractor acts as a wrapper around the model; when the model performs inference, the extractor captures the necessary activations. The user then specifies a class index to generate an activation map, which can be visualized as a raw heatmap or overlaid on the original image.

Who it’s for

It is designed for researchers and developers working with PyTorch-based computer vision models who need to interpret their model's decision-making process.

Highlights

• Extensive Method Zoo: Implements a wide range of CAM techniques including Grad-CAM, Grad-CAM++, Score-CAM, Layer-CAM, and LeGrad for Vision Transformers.
• Seamless Integration: Works as a wrapper around existing PyTorch models using hooks, requiring minimal code changes.
• Hugging Face Space: Includes a live demo app for testing different CAM methods.
• Performance Benchmarks: Provides detailed metrics on classification confidence and latency overhead for various methods.
• Broad Compatibility: Supports standard convolutional layers, Vision Transformers, 3D/video data, and batched inputs.