ml-stat-Sustech/TorchCP

A Python toolbox for conformal prediction research on deep learning models, using PyTorch.

What it solves

TorchCP provides a standardized toolbox for Conformal Prediction (CP) in deep learning. It addresses the challenge of quantifying uncertainty in model predictions by generating prediction sets (for classification) or intervals (for regression) that are guaranteed to contain the true label with a user-specified probability, regardless of the underlying model architecture.

How it works

Built on PyTorch and utilizing GPU acceleration, TorchCP implements a wide array of conformal prediction methods. It supports both post-hoc methods (applied after a model is trained) and training-time methods (integrating CP into the loss function or training process). The library is organized by task type, offering specialized score functions and predictors for different modalities.

Who it’s for

It is designed for researchers and practitioners in machine learning who need rigorous uncertainty quantification for deep learning models, particularly those working with classification, regression, graph neural networks, and large language models.

Highlights

  • Broad Task Support: Implements CP methods for classification, regression, Graph Neural Networks (GNNs), and LLMs.
  • Diverse Methodologies: Includes both post-hoc and training-integrated conformal methods.
  • GPU Accelerated: Leverages PyTorch for high-performance computation.
  • Extensive Implementation: Integrates numerous state-of-the-art methods from major venues like NeurIPS, ICML, and ICLR.
  • High Reliability: Maintains 100% unit test coverage.

Related

  • Project
    torch-uncertainty/torch-uncertainty

    A deep learning framework for uncertainty quantification that provides tools to make neural networks more reliable through uncertainty-aware training, evaluation, and post-processing.

  • Project
    deel-ai/puncc

    A Python library for predictive uncertainty calibration and conformalization, providing rigorous uncertainty estimations for any machine learning model across various tasks.

  • Project
    metaopt/torchopt

    An efficient PyTorch library for differentiable optimization that supports explicit, implicit, and zero-order differentiation for bi-level optimization and meta-learning.

  • Project
    jacobgil/pytorch-grad-cam

    A PyTorch library for explainable AI in computer vision that provides a wide range of pixel attribution methods to visualize and diagnose model predictions.

  • Project
    TorchSpatiotemporal/tsl

    A PyTorch-based library for neural spatiotemporal forecasting that provides a unified framework for data preprocessing, model prototyping, and Graph Neural Network implementation.