StatMixedML/XGBoostLSS

What it solves

Standard regression models typically predict a single point estimate (like the mean). XGBoostLSS extends XGBoost to perform probabilistic modelling, allowing users to predict the full conditional distribution of targets. This enables the creation of probabilistic forecasts, including the derivation of prediction intervals and specific quantiles of interest.

How it works

It provides a framework that models targets as a function of covariates using a wide variety of continuous, discrete, and mixed discrete-continuous distributions. The system leverages PyTorch for the automatic derivation of Gradients and Hessians for all distributional parameters and integrates with Pyro. It also supports complex data characteristics through Normalizing Flows (for multi-modal distributions), Mixture-Densities, and Zero-Adjusted/Zero-Inflated distributions for data with an excess of zeros.

Who it’s for

Data scientists and researchers who need more than a point prediction and require a full understanding of the uncertainty and distribution of their target variables in univariate or multivariate regression tasks.

Highlights

• Full Distributional Estimation: Estimates all parameters of the chosen distribution rather than just the mean.
• Multi-target Support: Capable of modelling multivariate responses and their dependencies.
• Flexible Distribution Library: Supports a broad range of distributional families, including mixture densities and normalizing flows.
• Integrated Tooling: Includes automated hyper-parameter search via Optuna and model explainability using SHAP.