vgvassilev/clad

What is Clad?

Clad is a C++ source‑transformation library that adds automatic differentiation (AD) to ordinary C++ code. It works as a Clang compiler plugin, analysing the source of a function and generating new C++ code that computes its derivatives. Both forward‑mode and reverse‑mode AD are supported, plus higher‑order utilities such as Hessians, Jacobians and floating‑point error estimation.

Why it matters for AI/ML

Training neural networks and many scientific‑machine‑learning models relies on gradients (and sometimes higher‑order derivatives). Most frameworks compute these gradients at runtime via interpreter‑level tricks. Clad instead generates native C++ derivative code at compile time, giving you:

Zero runtime overhead compared to hand‑written derivative code.
Full C++ performance – the generated functions are ordinary C++ functions that the optimizer can inline and vectorise.
Compatibility with existing C++ projects – you keep your codebase in plain C++ and just plug Clad into the build.

Core API (what you actually call)

Function	Mode	What it returns	Typical use
`clad::differentiate(f, arg)`	Forward	Functor whose `execute` runs the derivative of `f` w.r.t. a single variable (`arg`).	Compute a directional derivative or a scalar gradient when the function has one output.
`clad::gradient(f, args?)`	Reverse	Functor that fills gradient variables passed by pointer.	Efficiently obtain all partial derivatives of a scalar‑output function.
`clad::hessian(f, args?)`	Forward + Reverse	Functor that fills a flattened Hessian matrix (row‑major) via a pointer argument.	Needed for second‑order optimisation or uncertainty quantification.
`clad::jacobian(f, args?)`	Vectorised forward	Functor that fills `clad::matrix` objects for each pointer/array output.	Compute the full Jacobian of vector‑valued functions.
`clad::estimate_error(f)`	Reverse	Functor like `gradient` plus an extra `double&` that receives a floating‑point error estimate.	Analyse numerical stability of a computation.

All generated functors expose two useful methods:

execute(...) – runs the derivative code with the same arguments you would give to the original function (plus extra output pointers).
dump() – prints the generated C++ source so you can inspect or copy‑paste it.

Getting started

1. Install (conda is the easiest)

conda install -c conda-forge clad

The package ships the Clad headers, the compiled libclad.so plugin and a matching Clang version (LLVM 12‑22 are supported).

2. Compile a program with Clad

clang++ -std=c++17 -I $CONDA_PREFIX/include \
        -fplugin=$CONDA_PREFIX/lib/libclad.so my_program.cpp

Add -fplugin-arg-clad--fdump-derived-fn if you want the generated derivative source printed automatically.

3. Minimal example (forward mode)

#include "clad/Differentiator/Differentiator.h"
#include <iostream>

double f(double x, double y) { return x * y; }

int main() {
    auto df_dx = clad::differentiate(f, "x");   // derivative wrt x
    std::cout << df_dx.execute(3.0, 4.0) << '\n'; // prints 4 (∂f/∂x)
    df_dx.dump();                                 // shows the generated code
}

Run it with the clang command above and you’ll see the derivative computed at compile time.

More advanced usage

Reverse‑mode gradient – call clad::gradient(f) and provide a pointer for each input to receive its derivative.
Hessian – auto h = clad::hessian(f); double H[4]; h.execute(x, y, H);
Jacobian of vector‑valued functions – allocate clad::matrix<double> objects for each output array and pass them to the functor.
Floating‑point error estimation – auto err = clad::estimate_error(f); … err.execute(..., total_error);

All of these follow the same pattern: label the original function, execute the generated functor, and optionally dump the source.

Development & contribution

Build from source – clone the repo, then use CMake with a matching LLVM/Clang installation (the README lists exact commands for Linux, macOS, and WSL2). The build produces libclad.so and a test suite (make check-clad).
Contribute – the project follows the usual LLVM/Clang contribution workflow (pull requests, clang‑format, lit tests). Documentation lives at https://clad.readthedocs.io.

When to reach for Clad

You already write performance‑critical C++ code (e.g., physics simulations, finance, scientific computing) and need gradients without pulling in a heavyweight Python‑based framework.
You want compile‑time derivative generation so the resulting binary has no AD runtime library.
You need higher‑order derivatives (Hessian, Jacobian) and would rather have the compiler generate them than hand‑code them.

If those points match your needs, Clad gives you a clean, header‑only‑style API that integrates directly with the Clang toolchain.