soda-inria/tabicl

What it solves

TabICL is designed to eliminate the need for tedious hyperparameter tuning in tabular classification and regression tasks. It provides a competitive, high-performance alternative to traditional gradient-boosted decision trees (like XGBoost or CatBoost) that works out-of-the-box without manual optimization.

How it works

It is a tabular foundation model based on a Transformer architecture. Instead of traditional training, it uses in-context learning (ICL), where the model performs a single forward pass using the training data as context to predict outcomes for test data. It was pre-trained on millions of synthetic datasets to acquire general learning capabilities. For increased speed on repeated inferences, it employs KV caching and supports CPU/disk offloading for very large datasets.

Who it’s for

Data scientists and ML engineers working with tabular data who want high accuracy without the time-consuming process of hyperparameter tuning, as well as researchers interested in tabular foundation models.

Highlights

• Zero-shot performance: Outperforms heavily tuned XGBoost, CatBoost, or LightGBM on approximately 80% of TabArena datasets.
• High speed: Up to 10x faster than TabPFN-2.5 on specific benchmarks (e.g., 50k samples, 100 features on H100 GPU).
• Scalability: Handles datasets from 300 to 100,000 training samples and up to 2,000 features, with support for even larger sets via offloading.
• C-compliant: Scikit-learn compliant API for easy integration into existing pipelines.
• Versatility: Supports classification, regression, and zero-shot time series forecasting.
• Extensibility: Includes tools for fine-tuning on specific datasets and full pre-training recipes.