IBM/AutoMLPipeline.jl

A package that makes it trivial to create and evaluate machine learning pipeline architectures.

What it solves

AutoMLPipeline (AMLP) simplifies the creation and optimization of complex machine learning pipelines. It addresses the "Pipeline Optimization Problem" (POP), where the user must simultaneously determine the best sequence of preprocessing steps (feature extraction, transformation, and selection) and the most effective model and hyperparameters for regression or classification tasks.

How it works

AMLP uses Julia's macro programming to allow users to define ML workflows using symbolic expressions. It employs a concise syntax where |> represents a sequence of operations and + represents the concatenation of parallel feature processing paths.

For example, a user can define a pipeline that processes categorical features via one-hot encoding and numerical features via PCA, then combines them before passing them to a Random Forest model. The package provides wrappers for popular libraries like Scikit-learn and DecisionTree, and allows for easy extension by overloading the fit! and transform! interfaces.

Who it’s for

It is designed for data scientists and ML practitioners who need to rapidly prototype, experiment with, and optimize the structure of their machine learning pipelines for classification and regression.

Highlights

  • Symbolic Pipeline API: High-level description of complex workflows using simple mathematical-style expressions.
  • Extensible Architecture: Easy to add new components by implementing two main interfaces.
  • Meta-ensembles: Supports recursive composition of ensembles for more robust predictions.
  • Feature Selectors: Built-in tools for separating categorical and numerical features for specialized preprocessing.
  • Broad Integration: Includes wrappers for Scikit-learn and other ML libraries.

Related

  • Project
    astroautomata/SymbolicRegression.jl

    A Julia library for symbolic regression that searches for human-readable mathematical expressions to fit datasets, optimizing the trade-off between accuracy and complexity.

  • Project
    SciML/ModelingToolkit.jl

    A modeling framework for high-performance symbolic-numeric computation used in scientific computing and scientific machine learning to optimize and solve complex equation-based models.

  • Project
    JuliaAI/MLJ.jl

    MLJ is a machine learning toolbox for Julia that provides a common interface and meta-algorithms for selecting, tuning, and comparing over 200 machine learning models.

  • Project
    mljar/mljar-supervised

    An Automated Machine Learning package for tabular data that automates preprocessing, model selection, and tuning while providing transparent, detailed reports for explainability.

  • Project
    Pipelex/pipelex

    A framework for building and running reusable, typed AI procedures called Methods, enabling declarative and repeatable AI pipelines across multiple LLM providers.