jax-ml/ml_dtypes

What it solves

It provides a stand-alone implementation of specialized numerical data types (dtypes) that are commonly used in machine learning but are not natively supported by NumPy. This allows ML libraries to use low-precision formats to reduce memory usage and increase computational efficiency.

How it works

The library implements several NumPy dtype extensions, registering them so they can be used directly within NumPy arrays. It supports a wide range of formats, including:

• Bfloat16: A truncated single-precision float.
• 8-bit Floats: Various configurations of exponent and mantissa bits (e.g., float8_e4m3, float8_e5m2).
• Microscaling (MX) formats: Sub-byte floating point representations (4-bit and 6-bit).
• Narrow Integers: 1, 2, and 4-bit integer types (stored as unpacked bytes).
• Complex types: 16-bit complex floating point numbers (complex32 and bcomplex32).

Who it’s for

Developers of machine learning libraries and researchers who need to use low-precision numerical formats for training or inference without needing to rewrite NumPy's core engine.

Highlights

• NumPy Integration: Registers dtypes with NumPy, allowing them to be referred to by their string names.
• Broad Format Support: Implements a diverse array of 8-bit, 4-bit, and 6-bit floating point and integer formats.
• Low-Precision Handling: Provides guidance and tools to manage precision loss during arithmetic operations, such as recommending higher-precision accumulation.