Easily Build and Share ROCm Kernels with Hugging Face

Hugging Face has released a guide and tooling to simplify the creation and distribution of ROCm-compatible kernels. By leveraging the kernels library and kernel-builder, developers can build high-performance GPU operations for AMD hardware and share them via the Hugging Face Hub, ensuring reproducibility and seamless PyTorch integration.

The RadeonFlow GEMM Kernel Example

To demonstrate the build process, Hugging Face uses the RadeonFlow GEMM kernel, a high-performance FP8 block-wise matrix multiplication implementation optimized for the AMD Instinct MI300X GPU.

Technical Specifications

  • Precision: Uses the e4m3fnuz FP8 floating-point format for inputs to increase throughput and reduce memory bandwidth.
  • Accuracy: Employs per-block scaling factors (a_scale and b_scale) to maintain numerical stability despite the limited dynamic range of FP8.
  • Inputs/Outputs:
    • a: K × M in e4m3fnuz
    • b: K × N in e4m3fnuz
    • a_scale: (K // 128) × M in fp32
    • b_scale: (K // 128) × (N // 128) in fp32
    • c: M × N in bf16
  • Recognition: This kernel won the Grand Prize in the AMD Developer Challenge 2025 in June 2025.

Building ROCm Kernels with kernel-builder

Developing custom kernels often involves complex build flags and ABI issues. The Hugging Face kernels library abstracts this complexity through a structured project organization and the use of Nix for reproducibility.

Project Structure

Projects are organized into specific directories to help the builder identify file types:

  • build.toml: The project manifest that orchestrates the build process.
  • gemm/: Contains raw HIP source code (.hip for implementations and .h for headers).
  • flake.nix: Ensures a reproducible build environment by locking dependencies.
  • torch-ext/: Contains the C++ bindings and Python wrappers needed to expose the kernel as a PyTorch operator.

Configuration and Registration

  • build.toml: Defines the backend (e.g., rocm), target architectures (e.g., gfx942 for MI300 series), and source files.
  • PyTorch Integration: Kernels are registered as native PyTorch operators using TORCH_LIBRARY_EXPAND. This allows the kernel to be accessed via torch.ops and behave as a first-class part of the PyTorch framework.
  • Python Wrapper: An __init__.py file provides a user-friendly interface, handling tensor creation and shape validation before calling the underlying operator.

Reproducibility and Deployment

Nix-Based Build Process

Building is handled via Nix to ensure that the environment is identical across different machines.

  • Locking: nix flake update generates a flake.lock file to pin the kernel-builder and its dependencies.
  • Caching: The Hugging Face cache (via cachix) is used to avoid expensive rebuilds of PyTorch versions.
  • Multi-version Support: The nix build . -L command can automate building the kernel for all supported versions of PyTorch and ROCm.

Distribution via Hugging Face Hub

Once built, kernels are uploaded to the Hugging Face Hub using the kernels upload command or via Git Xet for binary files (.so files). This removes the need for traditional installation; users can load the kernel directly from the Hub using get_kernel:

import torch
from kernels import get_kernel

# Load the kernel from the Hub
gemm = get_kernel("kernels-community/gemm")

# Execute the kernel
result = gemm.gemm(A_fp8, B_fp8, A_scale, B_scale, C)

Related Resources

  • kernels library: The core library for building, managing, and loading kernels.
  • Kernels Community Hub: A central repository for discovering and sharing community-created kernels.

Sources