StatMixedML/LightGBMLSS

An extension of LightGBM to probabilistic modelling

What it solves

LightGBMLSS extends LightGBM to enable probabilistic modelling. While standard gradient boosting machines often predict a single point estimate, LightGBMLSS allows users to model and predict the full conditional distribution of a target variable based on covariates. This enables the creation of probabilistic forecasts, allowing for the derivation of prediction intervals and specific quantiles of interest.

How it works

The framework is built on PyTorch and Pyro, leveraging PyTorch's autograd to automatically derive gradients and Hessians for all distributional parameters. It supports a wide range of distribution families, including continuous, discrete, and mixed discrete-continuous distributions, as well as more complex types like Normalizing Flows and Mixture-Densities to handle multi-modal distributions. It also includes support for Zero-Inflated and Zero-Adjusted distributions to handle data with an excess of zeros.

Who it’s for

Data scientists and researchers who need more than a point estimate and require probabilistic forecasts and uncertainty quantification in their predictive models using the LightGBM ecosystem.

Highlights

  • Full Distributional Estimation: Estimates all parameters of a chosen distribution to predict the full conditional distribution.

  • Cores PyTorch Integration: Uses PyTorch for automatic derivation of Gradients and Hessians.

  • Diverse Distribution Support: Supports Normalizing Flows, Mixture-Densities, and Zero-Inflated/Adjusted distributions.

  • Ecosystem Compatibility: Maintains full compatibility with all LightGBM features and functionality.

  • Integrated Tooling: Includes automated hyper-parameter search via Optuna and model explanation via SHAP.

Related

  • Project
    StatMixedML/XGBoostLSS

    An extension of XGBoost that enables probabilistic modelling by predicting the full conditional distribution of targets instead of single point estimates.

  • Project
    lightgbm-org/LightGBM

    A highly efficient gradient boosting framework that uses tree-based learning algorithms to handle large-scale data with high speed and low memory usage.

  • Project
    awslabs/gluonts

    A Python package for probabilistic time series modeling that uses deep learning and PyTorch to generate forecasts with uncertainty intervals.

  • Project
    fabsig/GPBoost

    A software library that combines tree-boosting with Gaussian processes and mixed-effects models to improve prediction accuracy for spatial data and high-cardinality categorical variables.

  • Project
    jacobgil/pytorch-grad-cam

    A PyTorch library for explainable AI in computer vision that provides a wide range of pixel attribution methods to visualize and diagnose model predictions.