libxsmm/libxsmm

Library for specialized dense and sparse matrix operations, and deep learning primitives.

What it solves

LIBXSMM is a high-performance library designed for small dense and sparse linear algebra operations, specifically targeting the matrix multiplications (GEMM) and element-wise primitives often used in deep learning and High Performance Computing (HPC) workloads. It addresses the performance gap for small matrix operations where traditional large-scale libraries may be lack efficiency.

How it works

The library uses Just-In-Time (JIT) code specialization to generate optimized kernels for specific matrix shapes and data types, ensuring compiler-independent performance. It serves as the reference implementation of Tensor Processing Primitives (TPP), providing a portable abstraction for operations like convolutions, fully-connected layers, and pooling. It supports a vast array of hardware architectures, including Intel (SSE, AVX, AVX-512, AMX), AArch64 (NEON, SVE, SME), RISC-V (RVV), and PowerPC 64-bit.

Who it’s for

It is intended for developers building deep learning frameworks, HPC applications, and anyone requiring highly optimized, architecture-portable small matrix operations across diverse CPU architectures.

Highlights

  • Broad Hardware Support: Targets Intel, ARM, RISC-V, and PowerPC architectures.
  • Extensive Data Type Support: Supports a wide range of precisions from FP64 down to int1, including bfloat16 and various low-precision formats.
  • JIT Specialization: Uses JIT code generation to optimize performance for specific problem sizes without requiring target-specific build flags.
  • TPP Reference: Acts as the reference implementation for Tensor Processing Primitives, enabling the composition of higher-level deep learning operators.

Related

  • Project
    deepseek-ai/DeepGEMM

    A high-performance CUDA kernel library for LLMs that optimizes GEMM and MoE operations, featuring runtime JIT compilation and extreme throughput on NVIDIA SM90/SM100 GPUs.

  • Project
    patrick-kidger/lineax

    A JAX library for linear solves and linear least squares, providing numerically stable routines for solving linear systems, including support for linear operators and GPU/TPU acceleration.

  • Project
    NVIDIA/TensorRT-LLM

    TensorRT‑LLM is NVIDIA’s open‑source library that compiles large language (and visual‑generation) models into ultra‑fast TensorRT inference engines. It provides custom CUDA kernels, quantization, sparse attention, speculative decoding, and multi‑GPU parallelism, delivering 2‑10× higher token throughput while keeping accuracy. The Python‑friendly runtime integrates with Triton, Kubernetes, and Jetson, and includes docs and examples for models like GPT‑OSS‑120B, Llama‑4, DeepSeek‑R1, and diffusion models.

  • Project
    google/jaxopt

    A library of hardware-accelerated, batchable, and differentiable optimizers built for JAX.

  • Project
    Simple-Robotics/proxsuite

    A collection of numerically robust and efficient numerical solvers for Linear and Quadratic Programs, designed for high-performance robotics and optimization tasks.