sapientml/sapientml

What it solves

SapientML addresses the "search space explosion" problem in Automated Machine Learning (AutoML). Traditional AutoML tools often struggle to find optimal pipelines or fail entirely when dealing with large, complex tabular datasets because the number of possible combinations of ML components is too vast to search efficiently.

How it works

SapientML uses a generative approach to synthesize machine learning pipelines by learning from a corpus of existing datasets and human-written solutions (mined from sources like Kaggle). It employs a three-stage divide-and-conquer program synthesis strategy:

1. Meta-learning: Predicts a set of plausible ML components for the specific task.
2. Pipeline Refinement: Uses a pipeline dataflow model to narrow these components down into a small pool of viable concrete pipelines.
3. Dynamic Evaluation: Evaluates the remaining pipelines to select the best-performing solution.

Who it’s for

It is designed for data scientists and developers who need to generate high-quality predictive pipelines for tabular data (classification and regression) without manually designing the entire workflow.

Highlights

• Generative Code: Unlike some AutoML tools that only provide a model object, SapientML can generate the actual Python code used to build the pipeline, which can be saved and reused.
• Human-Inspired: Learns from a large corpus of 1,094 human-written pipelines across 170 datasets.
• Robustness: Specifically designed to handle large, real-world datasets where other AutoML tools may fail to produce a pipeline.