uxlfoundation/oneDNN

What it solves

oneDNN is a cross-platform performance library that provides basic building blocks for deep learning applications. It solves the problem of optimizing deep learning workloads across different hardware architectures, ensuring that applications can run efficiently on a variety of CPUs and GPUs without needing to write hardware-specific code for every platform.

How it works

The library provides a set of optimized primitives for deep learning. It uses a CPU dispatcher that detects the instruction set architecture (ISA) at runtime and employs just-in-time (JIT) code generation to deploy the most optimized code for the latest supported ISA. It supports multiple runtimes (OpenMP, TBB, SYCL) and provides separate engines for CPU and GPU execution.

Who it’s for

It is designed for deep learning application and framework developers who want to improve the performance of their software on CPUs and GPUs. It is not intended for end-users or deep learning practitioners, who are instead encouraged to use frameworks enabled with oneDNN, such as PyTorch, TensorFlow, and llama.cpp.

Highlights

• Broad Hardware Support: Optimized for Intel 64/AMD64, Arm AArch64, and Intel Graphics, with experimental support for NVIDIA and AMD GPUs, RISC-V, and Power ISA.
• Cross-Platform: Works across Linux, Windows, and macOS.
• Framework Integration: Integrated into major deep learning frameworks like PyTorch, TensorFlow, and ONNX Runtime.
• JIT Optimization: Uses runtime ISA detection and JIT code generation for peak CPU performance.