dm-haiku: a neural network library for JAX that enables object-oriented model definition

What it solves

JAX is a powerful numerical computing library, but it requires functions to be pure to use its transformations (like jit and grad). Haiku solves this by allowing developers to use a familiar object-oriented programming model to define neural networks while automatically transforming those "impure" object-oriented definitions into pure functions that JAX can process.

How it works

Haiku provides two primary tools to bridge the gap between object-oriented design and functional purity:

• hk.Module: A Python object used to define network layers and components. These modules hold references to parameters and methods, allowing users to write code that looks like standard neural network libraries.
• hk.transform: A function transformation that converts a function using hk.Module into a pair of pure functions: init (which collects the initial parameter values) and apply (which injects those parameters back into the function for computation).

For models requiring internal mutable state (like batch normalization), Haiku provides hk.transform_with_state, which manages parameters and state separately.

Who it’s for

Researchers and developers who want the productivity of an object-oriented API for building neural networks but need the full power of JAX's function transformations and hardware acceleration.

Highlights

• DeepMind Scale: Tested by researchers at DeepMind for large-scale image, language, and reinforcement learning experiments.
• Library, Not Framework: Designed as a lightweight library that focuses on parameter and state management without imposing custom optimizers or checkpointing formats.
• hk.next_rng_key(): Simplifies random number generation within JAX by providing a deterministic sequence of keys.
• JAX Compatibility: Fully compatible with jax.pmap for distributed training across multiple accelerators.