arogozhnikov/einops

What it solves

einops provides a flexible and powerful way to perform tensor operations, making code more readable, reliable, and framework-independent. It replaces complex, often confusing reshapes, transpositions, and reductions with a clear, semantic notation that describes what the operation is doing rather than how it is computed.

How it works

The library uses an Einstein-inspired notation to describe tensor transformations. Instead of using framework-specific functions like .view() or .reshape(), users specify a pattern string (e.g., 'b c h w -> b (c h w)') to rearrange, reduce, or repeat elements.

Core operations include:

• Rearrange: Changes the layout of elements according to a pattern.
• Reduce: Combines rearrangement and reduction (e.g., using 'mean' or 'max') into a single step.
• Repeat: Copies elements along a new axis.
• Pack/Unpack: Reversibly packs multiple tensors into one, which is useful for handling different dimensionalities.
• Einsum: A flexible dot-product operation supporting multi-lettered names.

It also provides framework-specific layers (e.g., for PyTorch, TensorFlow, Jax) that can be integrated directly into model architectures as layers.

Who it’s for

Researchers and developers working with deep learning frameworks (such as PyTorch, TensorFlow, JAX, MLX, and NumPy) who frequently manipulate multi-dimensional tensors and want to avoid bugs and improve code maintainability.

Highlights

• Semantic Clarity: Patterns describe the input and output shapes, providing built-in documentation of the tensor's meaning.
• Framework Agnostic: Works consistently across NumPy, PyTorch, TensorFlow, JAX, MLX, and any framework supporting the Python array API standard.
• Uniformity: Provides a uniform way to handle 1D, 2D, and 3D pooling or complex operations like space-to-depth transformations.
• Reliable Checks: Allows specifying particular dimensions in the pattern, which automatically validates the input tensor's shape.
• Citing: An ICLR 2022 oral presentation paper is available for the project.