uxlfoundation/oneDNN

Open-source library of optimized deep learning operations (matmul, convolution, attention) for CPUs (x64, AArch64, RISC-V) and Intel GPUs. Used by PyTorch, TensorFlow, OpenVINO, and ONNX Runtime.

What it solves

oneDNN is a cross-platform library that provides highly optimized implementations of core deep learning operations. It solves the problem of hardware-specific performance bottlenecks by acting as an acceleration layer for AI frameworks, ensuring that operations like matrix multiplication and convolution run efficiently across different CPU and GPU architectures.

How it works

The library uses a single API to target multiple devices including Intel, AMD, and Arm CPUs, as well as various GPUs. It employs Just-In-Time (JIT) code generation to detect hardware features at runtime and generate kernels specialized for the specific instruction set, core count, and cache sizes of the processor.

Who it’s for

It is primarily designed for deep learning application and framework developers who want to maximize performance on CPUs and GPUs. It is used as a backend by major frameworks such as PyTorch, TensorFlow, and OpenVINO.

Highlights

  • Broad Hardware Support: Targets Intel64/AMD64, AArch64, RISC-V CPUs, and Intel GPUs, with experimental support for NVIDIA and AMD GPUs.
  • Low-Precision Inference: Supports a wide range of data types including f32, bf16, f16, fp8, fp4, and various integer formats.
  • Operation Fusion: Fuses activations, residual connections, and quantization into matmuls and convolutions, and supports larger patterns like scaled dot-product attention (SDPA) via its Graph API.
  • Interoperability: Compatible with OpenMP, TBB, SYCL, and OpenCL.

Related

  • Project
    NVIDIA/cudnn-frontend

    NVIDIA’s cuDNN Frontend is a header‑only C++ library plus Python bindings that simplify use of the cuDNN Graph API. It offers a unified graph abstraction, auto‑tuning, and a suite of open‑source high‑performance kernels (Flash‑Attention, fused GEMM‑+‑activation, sparse attention, etc.) for Hopper and Blackwell GPUs, supporting FP16/BF16/FP8. Install via pip or by including the header, and integrate directly with PyTorch or CUDA Graphs.

  • Project
    mil-tokyo/webdnn

    A browser-based neural network inference engine that runs ONNX models directly in the web browser using WebGPU, WebGL, and WebAssembly.

  • Project
    Samsung/ONE

    ONE is a high-performance on-device neural network inference framework that enables AI models from TensorFlow and PyTorch to run efficiently across various hardware accelerators.

  • Project
    TrenTorch/TrenTorch

    Tren⚡Torch is an educational, NumPy‑only deep‑learning framework that re‑implements TinyTorch and adds 20 progressive modules—from tensors and autograd to CNNs, transformers, quantization and benchmarking—plus a CLI and historic‑milestone scripts for hands‑on learning.

  • Project
    google/XNNPACK

    XNNPACK is a highly optimized library of low-level performance primitives used to accelerate neural network inference across ARM, x86, WebAssembly, and RISC-V platforms.