pyro-ppl/numpyro

What it solves

NumPyro is a probabilistic programming library designed to perform Bayesian inference on models. It solves the problem of slow inference speeds in probabilistic programming by leveraging JAX for automatic differentiation and Just-In-Time (JIT) compilation, allowing models to run efficiently on CPUs, GPUs, and TPUs.

How it works

NumPyro provides a NumPy-like backend for the Pyro probabilistic programming language. It uses JAX to compile the entire integration step of inference algorithms into XLA optimized kernels, reducing Python overhead. The library includes a comprehensive set of distribution classes, constraints, and bijective transforms, and supports effect handlers to implement custom inference algorithms.

Who it’s for

It is intended for researchers and developers who need to perform complex probabilistic modeling and Bayesian inference, particularly those already familiar with the Pyro or PyTorch distributions API.

Highlights

• JAX-powered performance: Uses JIT compilation and autograd to accelerate MCMC and variational inference on hardware accelerators.
• Cores MCMC algorithms: Implements the No-U-Turn Sampler (NUTS), Hamiltonian Monte Carlo (HMC), MixedHMC for discrete variables, and HMCECS for large datasets.
• Variational Inference: Supports Automatic Differentiation Variational Inference (ADVI) with flexible guides for models including those with discrete latent variables.
• Flexible Distributions: Provides a wide range of distribution classes and supports distributions from TensorFlow Probability (TFP).
• C++ like speed in Python: Eliminates Python overhead in the NUTS tree building stage through Iterative NUTS.