microsoft/hummingbird

What it solves

Hummingbird solves the problem of slow inference for traditional machine learning models (like decision trees and random forests) by allowing them to be executed as tensor computations. This enables these models to leverage the hardware acceleration (GPUs) and optimization techniques typically reserved for deep learning frameworks without requiring the user to re-engineer their original models.

How it works

It compiles trained traditional ML models into tensor computations by reconfiguring algorithmic operators into regular computations that are compatible with vectorized and GPU execution. For example, it can translate a decision tree's traversal into a series of matrix multiplications (GEMM strategy), converting the model's logic into tensors that a framework like PyTorch can process efficiently.

Who it’s for

It is designed for data scientists and ML engineers who use traditional ML libraries like scikit-learn, LightGBM, or XGBoost and want to accelerate their model inference using neural network backends such as PyTorch, TorchScript, ONNX, or TVM.

Highlights

• Framework Flexibility: Converts models to PyTorch, TorchScript, ONNX, and TVM.
• Broad Model Support: Compatible with scikit-learn Decision Trees and Random Forests, as well as LightGBM and XGBoost classifiers and regressors.
• Seamless Integration: Provides a uniform inference API that follows the scikit-learn pattern, allowing users to swap models without changing their inference code.
• Hardware Acceleration: Enables native GPU acceleration for traditional ML models via the target tensor framework.