torch-uncertainty/torch-uncertainty
Open-source framework for uncertainty and deep learning models in PyTorch 🌱
What it solves
Deep neural networks often provide predictions without indicating how certain the model is about those predictions. TorchUncertainty provides a comprehensive framework to quantify this uncertainty, making AI models more reliable and safer for deployment in critical applications.
How it works
The library implements a wide array of uncertainty quantification (UQ) techniques across four primary tasks: classification, probabilistic and pointwise regression, segmentation, and pixelwise regression. It integrates with PyTorch Lightning for streamlined training and evaluation routines.
It offers several layers of functionality:
- Models: Implementations of Deep Ensembles, MC-Dropout, Variational Bayesian Neural Networks, and Packed-Ensembles.
- Post-processing: Tools for calibration (e.g., temperature scaling) and conformal predictions.
- Evaluation: Automated metrics for out-of-distribution (OOD) detection, calibration (ECE), and selective classification.
- Data: Specialized augmentations like Mixup and corruptions to test model robustness under distribution shift.
Who it’s for
Researchers and developers building deep learning models who need to know when their model is likely to be wrong or how to handle out-of-distribution data.
Highlights
- Multi-task support: Works for classification, regression, and image segmentation.
- Official implementations: Includes official code from major ML/CV conferences (ICLR, TPAMI, BMVC).
- Comprehensive evaluation: Built-in tools for proper scores, calibration, and OOD detection.
- Lightning integration: Easy-to-use training and evaluation routines via PyTorch Lightning.