facebookresearch/nevergrad

What it solves

It provides a way to minimize functions without needing gradients, which is essential for optimizing hyperparameters or complex functions where the same mathematical derivative is not available.

How it works

The library uses a variety of gradient-free optimization algorithms to find the minimum value of a given function. It allows users to define a search space (parametrization) that can include continuous variables, integers, and categorical choices (e.g., "conv" vs "fc").

Who it’s for

Researchers and developers who need to optimize parameters for machine learning models or other mathematical functions where traditional gradient-based methods are not applicable.

Highlights

• Supports bounded continuous variables, discrete variables, and mixtures of different input types.
• Simple API for minimizing functions with a minimal amount of code.
• Flexible parametrization system for handling log-distributed scalars, integers, and choices.
• Gradient-free approach, making it suitable for a complex variety of optimization problems.