jax-ml/jax

What it solves

JAX provides a high-performance system for numerical computing and large-scale machine learning. It allows developers to write NumPy-like code that can be automatically differentiated, compiled for speed, and scaled across multiple hardware accelerators like GPUs and TPUs.

How it works

JAX uses XLA (Accelerated Linear Algebra) to compile Python and NumPy functions into optimized machine code. It operates as a system of composable function transformations:

• jax.grad: Computes gradients of Python and NumPy functions using automatic differentiation (supporting both reverse-mode and forward-mode).
• jax.jit: Uses XLA to compile functions end-to-end for faster execution.
• jax.vmap: Automatically vectorizes functions, mapping them across array axes to eliminate manual batching loops.

Who it’s for

It is designed for researchers and developers working in high-performance numerical computing and large-scale machine learning who need efficient gradient computation and the ability to scale computations across thousands of devices.

Highlights

• Composable Transformations: Combine grad, jit, and vmap in any order to create highly optimized functions.
• Hardware Acceleration: Native support for NVIDIA GPUs, Google TPUs, and other accelerators via XLA.
• Flexible Differentiation: Differentiates through loops, branches, recursion, and closures to any order.
• Scaling Options: Offers three modes of scaling: compiler-based automatic parallelization, explicit sharding, and manual per-device programming.