libxsmm/libxsmm

What it solves

LIBXSMM provides high-performance linear algebra operations specifically optimized for small dense and sparse matrices. It addresses the performance gap in deep learning and HPC workloads where standard libraries may not be efficient for very small matrix multiplications (SMM) or element-wise primitives.

How it works

The library uses Just-In-Time (JIT) code specialization to generate optimized kernels at runtime, ensuring compiler-independent performance. It serves as the reference implementation for Tensor Processing Primitives (TPP), a programming abstraction that allows higher-level operators like convolutions, pooling, and fully-connected layers to be composed from basic primitives. It supports a wide array of hardware architectures (Intel SSE/AVX/AMX, AArch64 NEON/SVE/SME, RISC-V RVV, and PowerPC POWER10) and a vast range of data types, from FP64 down to 1-bit integers.

Who it’s for

It is designed for developers working on deep learning frameworks, HPC applications, and any software requiring highly optimized, architecture-portable small matrix operations.

Highlights

• JIT Specialization: Generates optimized code on the fly for specific matrix shapes and types.
• Broad Hardware Support: Targets multiple CPU architectures including Intel, ARM, RISC-V, and PowerPC.
• Extensive Data Types: Supports a massive variety of precisions including FP32, BF16, INT8, and various low-precision formats (e.g., MXFP4).
• Architecture Portable: Designed for "build once, deploy everywhere" without needing special target flags.
• TPP Reference: Implements the Tensor Processing Primitives abstraction for portable deep learning workloads.