StatMixedML/XGBoostLSS

An extension of XGBoost to probabilistic modelling

What it solves

Standard regression models typically predict a single point value (the mean or median). XGBoostLSS extends XGBoost to perform probabilistic modelling, allowing users to predict the entire conditional distribution of a target variable. This enables the creation of probabilistic forecasts, including the derivation of prediction intervals and specific quantiles, rather than just a single estimate.

How it works

It provides a framework that models all distributional parameters of a target as a function of covariates. It leverages PyTorch for the automatic derivation of gradients and Hessians for these parameters and supports a wide variety of distribution types, including continuous, discrete, and mixed discrete-continuous distributions. For more complex data, it incorporates Normalizing Flows for multi-modal distributions and Mixture-Densities.

Who it’s for

Data scientists and researchers who need more than point predictions and require uncertainty quantification or the ability to model complex, non-standard data distributions (such as zero-inflated data) using the power of gradient boosting.

Highlights

  • Full Distributional Estimation: Predicts all parameters of a chosen probability distribution.
  • Diverse Distribution Support: Includes zero-adjusted and zero-inflated distributions for data with excess zeros.
  • Advanced Modelling: Supports Normalizing Flows and Mixture-Densities for complex distributions.
  • Multi-target Regression: Capable of modelling multivariate responses and their dependencies.
  • Tooling Integration: Integrates with Optuna for automated hyper-parameter search and SHAP for model explainability.

Related

  • Project
    StatMixedML/LightGBMLSS

    An extension of LightGBM that enables probabilistic modelling by predicting the full conditional distribution of a target variable instead of a single point estimate.

  • Project
    fabsig/GPBoost

    A software library that combines tree-boosting with Gaussian processes and mixed-effects models to improve prediction accuracy for spatial data and high-cardinality categorical variables.

  • Project
    dmlc/xgboost

    XGBoost is an optimized distributed gradient boosting library that provides fast and accurate parallel tree boosting for large-scale data science problems.

  • Project
    awslabs/gluonts

    A Python package for probabilistic time series modeling that uses deep learning and PyTorch to generate forecasts with uncertainty intervals.

  • Project
    Nixtla/hierarchicalforecast

    A library for probabilistic hierarchical forecasting that ensures consistency across different levels of data aggregation using statistical and econometric reconciliation methods.