gfo-project/Gradient-Free-Optimizers

What it solves

Gradient-Free-Optimizers (GFO) provides a unified way to optimize "black-box" functions where gradients are unavailable or impractical to calculate. It is specifically designed for tasks like hyperparameter tuning for machine learning models, simulation optimization, and engineering design where the relationship between inputs and outputs is not mathematically defined by a derivative.

How it works

The library provides a consistent API for 23 different optimization algorithms, categorized into local search, global search, population-based, and sequential model-based techniques (such as Bayesian optimization). Users define an objective function to maximize and a search space that can mix continuous ranges, discrete grids, categorical choices, and SciPy distributions. The optimizer then proposes parameters, evaluates the objective function, and iterates to find the best score.

Who it’s for

Data scientists, ML engineers, and researchers who need to tune hyperparameters or optimize complex systems without relying on gradient-based methods. It is also suitable for developers working in minimal environments, containers, or embedded systems due to its low dependency footprint (requiring only NumPy and pandas).

Highlights

• Unified API: Switch between 23 different algorithms (e.g., Hill Climbing, Bayesian, Particle Swarm) with a single line of code.
• Mixed Search Spaces: Support for combining continuous, discrete, categorical, and distribution-backed dimensions in one search.
• Built-in Caching: A memory system prevents redundant evaluations of expensive objective functions.
• Flexible Control: Includes an "Ask/Tell" interface for manual loop control, early stopping criteria, and constraint support to avoid invalid regions.
• Lightweight: Minimal core dependencies, making it easy to integrate as a backend for other tools.