meta-pytorch/KernelAgent

What it solves

KernelAgent automates the creation and performance tuning of GPU kernels. It solves the difficulty of manually writing high-performance Triton kernels by converting PyTorch programs into verified, optimized Triton code, reducing the need for expert-level GPU programming knowledge to achieve hardware-specific performance gains.

How it works

The system operates through two primary pipelines:

1. Kernel Generation: It uses an "AutoRouter" to analyze PyTorch ASTs and decide on the best path. A "Fuser Orchestrator" refactors the code into fusable subgraphs, which are then dispatched to parallel LLM-powered agents. These agents iteratively generate Triton kernels and verify them against unit tests. Finally, a "Composer" stitches these kernels into a single, verified Triton program.

2. Kernel Optimization: This pipeline uses a hardware-guided loop. It profiles kernels using NVIDIA NCU to collect hardware metrics, performs roofline analysis to identify if a kernel is memory-bound or compute-bound, and uses an LLM to diagnose bottlenecks and suggest fixes. The loop continues until the kernel reaches a performance ceiling (Speed-of-Light) or converges.

Who it’s for

• GPU developers and ML engineers who want to optimize PyTorch operations without writing raw Triton kernels by hand.
• Researchers working on custom operator fusion and hardware-specific kernel tuning for NVIDIA and Intel XPU GPUs.

Highlights

• Multi-Platform Support: Works with both NVIDIA CUDA and Intel XPU GPUs.
• Multi-Agent Orchestration: Uses parallel workers to generate and refine kernels based on strict runtime verification.
• Hardware-Guided Tuning: Integrates real GPU profiling (NCU) and roofline analysis into the LLM optimization loop.
• End-to-End Automation: Handles everything from PyTorch code analysis and subgraph extraction to final composition and benchmarking.