deel-ai/puncc

👋 Puncc is a python library for predictive uncertainty quantification using conformal prediction.

What it solves

Puncc provides a way to transform point predictions from any machine learning model into prediction intervals or sets with rigorous uncertainty estimations. It ensures that the true output is covered within a user-defined error rate (alpha), provided the data is exchangeable (i.i.d.).

How it works

The library implements a variety of state-of-the-art conformal prediction algorithms. It wraps existing predictive models to compute nonconformity scores on a calibration dataset and then uses these scores to generate prediction intervals (for regression) or prediction sets (for classification) for new data.

Who it’s for

Data scientists and ML engineers who need reliable uncertainty quantification for their models, regardless of the model architecture (compatible with Scikit-learn, PyTorch, and TensorFlow/Keras).

Highlights

  • Broad Task Support: Supports regression, classification, object detection, and anomaly detection.
  • Model Agnostic: Works with any predictive model via a wrapper API.
  • Diverse Algorithms: Includes implementations of SplitCP, CQR, APS, RAPS, and others.
  • Flexible API: Offers both a high-level direct approach for standard procedures and a low-level API for custom nonconformity scores and calibration logic.
  • Evaluation Tools: Built-in metrics and plotting capabilities to evaluate conformal procedures.

Related

  • Project
    scikit-learn-contrib/MAPIE

    A model-agnostic Python library for quantifying uncertainty and controlling risks in machine learning models using conformal prediction.

  • Project
    ml-stat-Sustech/TorchCP

    A PyTorch-based toolbox for conformal prediction in deep learning, providing guaranteed uncertainty quantification for classification, regression, GNNs, and LLMs.

  • Project
    torch-uncertainty/torch-uncertainty

    A deep learning framework for uncertainty quantification that provides tools to make neural networks more reliable through uncertainty-aware training, evaluation, and post-processing.

  • Project
    sepandhaghighi/pycm

    A multi-class confusion matrix library for Python that provides extensive statistical metrics and visualization tools for evaluating classification models.

  • Project
    Nixtla/hierarchicalforecast

    A library for probabilistic hierarchical forecasting that ensures consistency across different levels of data aggregation using statistical and econometric reconciliation methods.