StatMixedML/LightGBMLSS

An extension of LightGBM to probabilistic modelling

What it solves

LightGBMLSS extends LightGBM to enable probabilistic modelling. Instead of predicting a single point estimate (like a mean or median), it predicts the full conditional distribution of a target variable based on covariates. This allows users to derive prediction intervals and quantiles, providing a more complete picture of uncertainty in forecasts.

How it works

The framework uses PyTorch and Pyro to model a wide range of continuous, discrete, and mixed discrete-continuous distributions. It automatically derives the Gradients and Hessian of all distributional parameters using PyTorch's autograd, which are then used by LightGBM to optimize the objective function. It also supports advanced modelling techniques such as Normalizing Flows for complex multi-modal distributions and Mixture-Densities for diverse data characteristics.

Who it’s for

Data scientists and researchers who need more than a point prediction and require probabilistic forecasts with associated uncertainty estimates for univariate target variables.

Highlights

  • Broad Distribution Support: Supports a wide range of distributions, including Zero-Adjusted and Zero-Inflated distributions for data with excess zeros.
  • Automatic Differentiation: Uses PyTorch to automatically calculate Gradients and Hessians for distributional parameters.
  • Advanced Modelling: Includes Normalizing Flows and Mixture-Densities for complex data distributions.
  • Integrated Tooling: Integrates with Optuna for automated hyper-parameter search and SHAP for model explainability.
  • LightGBM Compatibility: Maintains full compatibility with all existing LightGBM features and functionality.

Related

  • Project
    StatMixedML/XGBoostLSS

    An extension of XGBoost that enables probabilistic modelling by predicting the full conditional distribution of targets instead of just point estimates.

  • Project
    lightgbm-org/LightGBM

    A highly efficient gradient boosting framework that uses tree-based learning algorithms to handle large-scale data with high speed and low memory usage.

  • Project
    stanfordmlgroup/ngboost

    A Python library implementing Natural Gradient Boosting for probabilistic prediction, allowing models to output probability distributions instead of single point estimates.

  • Project
    awslabs/gluonts

    A Python package for probabilistic time series modeling that uses deep learning and PyTorch to generate forecasts with uncertainty intervals.

  • Project
    TorchDSP/torchsig

    An open-source signal processing machine learning toolkit based on PyTorch that simplifies the creation, augmentation, and transformation of RF signals for ML research.