Hugging Face CUDA Kernels Agent Skill

Hugging Face has developed an agent skill that teaches coding agents how to write production-ready CUDA kernels. By providing domain-specific knowledge to agents like Claude and Codex, the team successfully generated working kernels for both a diffusers pipeline and a transformers model, complete with PyTorch bindings and performance benchmarks.

Enabling Agents to Write CUDA Kernels

Writing CUDA kernels is traditionally a high-barrier task requiring deep knowledge of GPU architecture-specific memory access patterns, vectorization strategies, and warp shuffle reductions. The Hugging Face agent skill bridges this gap by packaging specialized domain expertise into a structured format that agents can load on demand.

Skill Components

The skill consists of approximately 550 tokens of structured guidance along with a comprehensive set of reference materials:

  • GPU Optimization Guides: Architecture-aware guidance for NVIDIA H100, A100, and T4 GPUs, covering compute capabilities, shared memory sizes, and bandwidth profiles.
  • Integration Patterns: Specific guides and pitfalls for integrating kernels into the diffusers and transformers libraries.
  • Kernel Templates: Vectorized memory access patterns for BF16, FP16, and FP32 precisions.
  • Workflows: Templates for isolated kernel micro-benchmarks and end-to-end pipeline comparisons.
  • Hub Integration: Instructions for using get_kernel to load community kernels from the HuggingFace Kernel Hub.

Installation and Usage

The skill is distributed via the kernels library. It can be installed into various agent environments (Claude Code, Cursor, Codex, OpenCode) using the kernels skills add cuda-kernels command. Once installed, users can prompt agents to build specific kernels, such as a "vectorized RMSNorm kernel for H100 targeting the Qwen3-8B model in transformers."

Performance Benchmarks

To validate the skill, Hugging Face tested the agent-generated kernels on two real-world targets using an H100 80GB HBM3 GPU at BFloat16 precision.

Diffusers: LTX-Video

The agent produced RMSNorm, RoPE 3D, GEGLU, and AdaLN kernels for the LTX-Video video generation pipeline.

  • Isolated RMSNorm Performance: The custom kernel achieved an average speedup of 1.88x over the PyTorch baseline, with bandwidth efficiency at 34.7% of the H100 theoretical maximum.
  • End-to-End Performance: The optimized kernels provided a 1.06x speedup (from 12.58 it/s to 13.52 it/s) compared to the baseline. When combined with torch.compile, the speedup increased to 1.43x.

Transformers: Qwen3-8B

The agent built an RMSNorm kernel for the Qwen3-8B LLM, which utilizes 65 RMSNorm modules.

  • Isolated RMSNorm Performance: The custom kernel achieved an average speedup of 1.94x over the PyTorch baseline.
  • Scaling: Speedup increased with sequence length, ranging from 1.58x at 128 tokens to 2.47x at 8192 tokens, effectively halving RMSNorm latency for long-context inference.

From Generation to Distribution via Kernel Hub

The agent skill handles the development phase, while the HuggingFace Kernel Hub handles distribution. The workflow for publishing an agent-generated kernel is as follows:

  1. Project Structure: The agent generates a project following the kernel-builder layout, including kernel_src/ (CUDA source), torch-ext/ (C++ bindings), and a build.toml configuration specifying the target GPU's CUDA capabilities (e.g., "9.0" for H100).
  2. Multi-Variant Build: Using a Nix flake, the developer builds the kernel for all required PyTorch and CUDA configurations to ensure compatibility across environment matrices.
  3. Hub Publication: The built binaries are uploaded to a model repository on the HuggingFace Hub.
  4. One-Line Loading: End users can load the pre-compiled kernel without local compilation using from kernels import get_kernel; rmsnorm = get_kernel("your-org/your-kernel").

Sources