aleximmer/Laplace

What is Laplace?

Laplace is a Python library that brings Bayesian uncertainty‑aware inference to any PyTorch neural network. It implements the Laplace approximation – a way of turning a trained deterministic model into a probabilistic one by fitting a Gaussian distribution around the network’s parameters. With the library you can:

Approximate the posterior over all weights, a chosen sub‑network, or just the last layer.
Estimate the marginal likelihood (useful for model comparison).
Produce predictive distributions for classification or regression, giving calibrated uncertainty estimates.

The package is tied to the research paper Laplace Redux – Effortless Bayesian Deep Learning (NeurIPS 2021) and the authors provide a separate repo with the experiments from that paper.

How do you use it?

Install – the core package is on PyPI under the name laplace-torch:
```
pip install laplace-torch
```
(An optional backend can be added with a second pip install command.)
Wrap a trained model – after you have a PyTorch model, create a Laplace object:
```
from laplace import Laplace
la = Laplace(model, "classification",
             subset_of_weights="all",
             hessian_structure="diag")
```
You tell the class what task you’re solving, which parameters to treat probabilistically, and how to approximate the Hessian (e.g., diagonal).
Fit the approximation – call fit with your training data loader. The library computes the (approximate) Hessian of the loss at the MAP point.

Tune the prior – the Gaussian prior’s precision is a hyper‑parameter. Laplace provides a few automatic schemes, such as a grid‑search cross‑validation:

la.optimize_prior_precision(method="gridsearch",
                            pred_type="glm",
                            link_approx="probit",
                            val_loader=val_loader)

Predict with uncertainty – once fitted, you can obtain predictive probabilities (or means/variances for regression) using a chosen predictive approximation:
```
pred = la(x, pred_type="glm", link_approx="probit")
```
The result is a distribution rather than a single point estimate, which you can use for calibrated confidence, out‑of‑distribution detection, model selection, etc.

The README points to extensive documentation and a developer guide for deeper customisation.

Who might find this useful?

Researchers exploring Bayesian deep learning, model‑selection, or continual‑learning scenarios who need a ready‑made, well‑tested Laplace implementation.
Practitioners who already have a PyTorch model and want to add uncertainty quantification without retraining a full Bayesian neural network.
Students learning about posterior approximations; the library’s API mirrors the mathematical steps (fit → optimise prior → predict), making experiments straightforward.

Limitations & practical notes

The approximation is not fully automatic – you need to choose a Hessian structure (diag, kron, full, etc.), a prior‑precision optimisation method, and a predictive approximation (GLM, MC, etc.). The README warns that performance can vary widely with these choices.
Computing a full Hessian is expensive; most users rely on diagonal or Kronecker‑factored approximations.
The library currently targets PyTorch only; other frameworks are not supported out of the box.

Quick start checklist

pip install laplace-torch
Train or load a PyTorch model.
Wrap it with Laplace(...) specifying task and Hessian style.
Call fit(train_loader).
Tune the prior with optimize_prior_precision (gridsearch is a safe default).
Use la(x, ...) to get predictive distributions.

Where to learn more?

Documentation & API reference: https://aleximmer.github.io/Laplace
Paper: Laplace Redux – Effortless Bayesian Deep Learning (NeurIPS 2021) – provides theory and empirical results.
Experiment code: https://github.com/runame/laplace-redux (reproduces the paper’s experiments).
Developer guide: https://aleximmer.github.io/Laplace/devs_guide (contributing, extending back‑ends, etc.)