torch-uncertainty/torch-uncertainty

What it solves

Deep neural networks often provide predictions without indicating how confident they are, which can lead to unreliable results in critical applications. TorchUncertainty provides a comprehensive framework to quantify this uncertainty, making models more reliable by helping users understand when a model is likely to be wrong.

How it works

The library provides a multi-level API that integrates with PyTorch and PyTorch Lightning to implement various uncertainty quantification techniques across four main tasks: classification, probabilistic/pointwise regression, segmentation, and pixelwise regression. It offers:

• Uncertainty-aware models: Implementations of Deep Ensembles, MC-Dropout, Variational Bayesian Neural Networks, and Deep Evidential models.
• Post-processing: Tools for calibration (e.g., temperature scaling) and conformal prediction to refine confidence estimates.
• Evaluation tools: Automated routines to measure proper scores, calibration, out-of-distribution detection, and distribution-shift metrics.
• Data Augmentation: Specialized methods like Mixup and modernized corruptions to test model robustness.

Who it’s for

It is designed for machine learning researchers and practitioners who need to move beyond simple point predictions and implement rigorous uncertainty estimation in their deep learning pipelines.

Highlights

• Broad Method Support: Includes official implementations of multiple papers from major ML and CV conferences.
• Task Versatility: Supports diverse tasks including image segmentation and monocular depth estimation.
• Automated Evaluation: Built-in metrics for OOD detection, selective classification, and calibration.
• Ready-to-use Components: Provides drop-in layers, models, metrics, and losses for easy integration into existing networks.