metaopt/torchopt

What it solves

TorchOpt provides a framework for differentiable optimization in PyTorch, enabling users to compute gradients through optimization processes. This is essential for bi-level optimization problems, such as meta-learning, where an outer-level parameter must be optimized based on the result of an inner-level optimization loop.

How it works

TorchOpt implements three primary differentiation modes to handle different optimization scenarios:

• Explicit Gradient (EG): Backpropagates through the unrolled optimization path, treating each gradient step as a differentiable function. This is ideal for a small number of inner-loop steps.
• Implicit Gradient (IG): Uses the implicit function theorem to find analytical derivatives at the stationary point of the inner-loop, avoiding the need to unroll the entire optimization path.
• Zero-order Differentiation (ZD): Estimates gradients using zero-order methods like finite-difference or Evolutionary Strategies (ES) when the inner-loop is non-differentiable or Hessian computations are too expensive.

It offers both a functional API (similar to JAX/Optax) and an object-oriented API (similar to standard PyTorch torch.optim) to suit different coding preferences.

Who it’s for

It is designed for researchers and developers working on meta-learning, hyperparameter optimization, and other bi-level optimization tasks who need an efficient, flexible way to differentiate through PyTorch optimizers.

Highlights

• Three Differentiation Modes: Supports explicit, implicit, and zero-order gradients.
• Flexible APIs: Provides both JAX-like functional and PyTorch-like object-oriented interfaces.
• Performance Optimizations: Includes C++/CUDA accelerated operators and an RPC-based distributed training framework.
• Functional Integration: Aligns with functorch to enable composable functional optimization in PyTorch.