jax-ml/jax

What it solves

JAX provides a high-performance system for numerical computing and large-scale machine learning. It solves the need for a library that can automatically differentiate Python and NumPy functions while scaling computations across hardware accelerators like GPUs and TPUs.

How it works

JAX is built as an extensible system of composable function transformations. It leverages XLA (Accelerated Linear Algebra) to compile NumPy-like programs for hardware accelerators. Its core functionality is driven by three primary transformations:

• jax.grad: Performs automatic differentiation, supporting both reverse-mode (backpropagation) and forward-mode differentiation to any order.
• jax.jit: Uses XLA to compile pure functions end-to-end for increased execution speed.
• jax.vmap: Automatically vectorizes functions, mapping them along array axes to eliminate the need for manual batch dimension management.

Who it’s for

JAX is designed for researchers and developers working in high-performance numerical computing and large-scale machine learning who need flexible, scalable array computations.

Highlights

• Composable Transformations: Ability to combine grad, jit, and vmap arbitrarily.
• Hardware Acceleration: Native support for NVIDIA GPUs, Google TPUs, AMD GPUs, and Intel GPUs via XLA.
• Flexible Differentiation: Supports differentiation through loops, branches, recursion, and closures.
• Scalable Parallelism: Offers multiple scaling modes, including compiler-based automatic parallelization, explicit sharding, and manual per-device programming.