Hugging Face kernel-builder: A Guide to Building and Scaling Production-Ready CUDA Kernels

Hugging Face has released the kernel-builder library, a toolset designed to streamline the creation, scaling, and deployment of custom CUDA kernels. By automating the build process across multiple PyTorch and CUDA versions and integrating with the Hugging Face Hub, kernel-builder allows developers to move from local GPU functions to robust, shareable production systems without the typical overhead of dependency management and build-time complexity.

Anatomy of a Modern CUDA Kernel

Building a production-ready kernel requires a structured approach to source code, build manifests, and environment reproducibility. Hugging Face recommends a specific project structure to ensure compatibility with the kernel-builder tool:

  • build.toml: The project manifest that orchestrates the build process.
  • csrc/: Contains the raw CUDA source code.
  • flake.nix: Ensures a reproducible build environment by locking the versions of the builder and its dependencies.
  • torch-ext/: Contains the Python wrappers and C++ bindings for PyTorch operators.

Registering Native PyTorch Operators

Rather than simple Python bindings, kernel-builder emphasizes registering functions as native PyTorch operators using the TORCH_LIBRARY_EXPAND macro. This approach provides two critical technical advantages:

  1. torch.compile Compatibility: Native registration allows PyTorch's compiler to "see" the operator, enabling it to be fused into larger computation graphs to reduce overhead.
  2. Hardware-Specific Implementations: Developers can provide multiple backends (e.g., CUDA and CPU) for the same operator. PyTorch's dispatcher automatically selects the correct implementation based on the input tensor's device.

The Build and Development Workflow

To ensure reproducibility, kernel-builder utilizes Nix shells. Developers can enter an isolated sandbox with specific versions of PyTorch and CUDA (e.g., nix develop .#devShells.torch27-cxx11-cu126-x86_64-linux) to compile and test kernels locally. The build2cmake command is used to generate the necessary CMake and Python build artifacts, allowing the kernel to be installed in editable mode via pip install -e . for rapid iteration.

Distribution and Scaling

To make a kernel available to the broader community, it must be "compliant," meaning it can be built and run across all supported versions of PyTorch and CUDA. The kernel-builder tool automates this multi-version build process via nix build . -L.

Hugging Face Hub Integration

Once built, kernels are uploaded to the Hugging Face Hub using the kernels upload command or via Git LFS. This enables users to load kernels dynamically without traditional installation:

from kernels import get_kernel
optimized_kernel = get_kernel("your-username/optimized-kernel")

Production Deployment Challenges

Scaling custom kernels in production requires strict versioning and deployment strategies to prevent breaking downstream dependencies.

Semantic Versioning and Locking

Because Hub repositories are Git-based, users can pin kernels to specific commit hashes. However, Hugging Face recommends semantic versioning (using Git tags like v1.1.2) to allow for graceful upgrades.

For large-scale projects, the kernels library supports project-level management via pyproject.toml. By specifying version bounds in the [tool.kernels.dependencies] section and running kernels lock ., developers generate a kernels.lock file. This ensures all users of a project utilize the exact same kernel versions, which can be loaded using get_locked_kernel.

Runtime and Deployment Optimization

To avoid downloading binaries at runtime—which is critical for Docker images and secure environments—the kernels library provides a pre-downloading mechanism:

  1. kernels download .: This CLI command downloads all kernels specified in the kernels.lock file into the local Hugging Face cache.
  2. load_kernel: This function loads the kernel from the local cache and raises an exception if the binary is missing, ensuring no unexpected network calls occur during application execution.

Legacy Support via Python Wheels

While Hub-based loading is preferred for its automatic version matching and provenance, kernel-builder supports legacy deployment via Python wheels. The kernels to-wheel command converts a Hub kernel into a set of wheels for various PyTorch, CUDA, and architecture combinations, allowing them to be distributed through traditional Python package managers.

Sources