fabsig/GPBoost

What it solves

GPBoost addresses the limitations of using either tree-boosting or latent Gaussian models in isolation. Classical tree-boosting often struggles with high-cardinality categorical variables and produces discontinuous predictions for spatial data. Conversely, traditional Gaussian processes (GP) and linear mixed effects models typically rely on linear prior mean functions, which can limit their prediction accuracy for complex, non-linear data.

How it works

GPBoost combines tree-boosting with latent Gaussian models (such as Gaussian processes and grouped random effects). It models the response variable as a combination of a non-linear mean function—represented by an ensemble of trees—and random effects.

For Gaussian likelihoods, it uses the GPBoost algorithm, while for non-Gaussian likelihoods, it uses the LaGaBoost algorithm. The library iteratively learns both the covariance parameters of the random effects and the predictor function (the tree ensemble) using functional gradient or Newton boosting steps.

Who it’s for

This library is designed for data scientists and researchers working with tabular data that includes spatial or spatio-temporal components, longitudinal/panel data, or high-cardinality categorical variables.

Highlights

• Hybrid Modeling: Integrates the non-parametric flexibility of tree-boosting with the probabilistic strengths of Gaussian processes.
• High-Cardinality Support: Efficiently handles categorical variables with many levels via grouped random effects.
• Spatial Continuity: Enables smooth, continuous predictions for spatial data, overcoming the "staircase" effect of standard decision trees.
• Multi-Language Support: Core implementation in C++ with accessible packages for both Python and R.