hyperopt/hyperopt

Distributed Asynchronous Hyperparameter Optimization in Python

What it solves

Hyperopt simplifies the process of finding the best hyperparameters for a machine learning model. It allows users to optimize "awkward" search spaces that include a mix of real-valued, discrete, and conditional dimensions, reducing the manual effort required to tune model performance.

How it works

Users define an objective function to minimize and a search space using the hp module. Hyperopt then applies optimization algorithms to suggest the best parameters. It supports three primary algorithms:

  • Random Search: A baseline approach for sampling parameters.
  • Tree of Parzen Estimators (TPE): A Bayesian optimization approach.
  • Adaptive TPE: An evolved version of TPE.

To handle large-scale tasks, the library can parallelize evaluations using Apache Spark or MongoDB.

Who it’s for

It is designed for data scientists and ML engineers who need to automate the tuning of complex model parameters across various types of data distributions.

Highlights

  • Flexible Search Spaces: Supports real-valued, discrete, and conditional dimensions.
  • Distributed Optimization: Integration with Apache Spark and MongoDB for parallel execution.
  • Multiple Algorithms: Includes TPE, Adaptive TPE, and Random Search.
  • Extensible Design: Built to accommodate Bayesian optimization algorithms.

Related

  • Project
    syne-tune/syne-tune

    A library for large-scale and reproducible hyperparameter optimization (HPO) that supports distributed execution and state-of-the-art tuning methods.

  • Project
    dmlc/xgboost

    XGBoost is an optimized distributed gradient boosting library that provides fast and accurate parallel tree boosting for large-scale data science problems.

  • Project
    python-adaptive/adaptive

    A Python library for parallel active learning of mathematical functions that intelligently selects the best points to sample in a parameter space to reduce computational cost.

  • Project
    NVIDIA/cuopt

    A GPU-accelerated optimization engine for solving large-scale linear, quadratic, and vehicle routing problems in near real-time.

  • Project
    automl/SMAC3

    A versatile Bayesian Optimization package designed to efficiently find optimal hyperparameter configurations for machine learning algorithms.