Dogacel/auto-gpu-kernel

What it solves

It automates the discovery and optimization of GPU kernels, which is typically a manual, high-effort engineering task. It prevents common AI agent failures, such as reacting to noisy performance measurements or getting stuck in local minima, by using a robust validation and benchmarking pipeline.

How it works

The system uses an optimization agent (kopt) and a validation/benchmarking tool (kbench). It can operate in two modes: using the flashinfer format for specific kernel generation or a custom format that allows it to target any Git repository. For arbitrary repositories, a setup agent first inspects the code and generates a custom benchmark and validation harness (harness/validate.py and harness/benchmark.py) based on plain-language descriptions of the objective and measurement criteria. The agent then iteratively optimizes the code, with kbench providing normalized measurements and A/B testing to ensure real speedups.

Who it’s for

GPU kernel engineers and AI researchers who want to automate the performance tuning of low-level GPU code or speed up inference in existing repositories without manually writing benchmarks.

Highlights

• Proven Performance: Ranked #1 in the MLSys 2026 FlashInfer AI Kernel Generation Contest for DeepSeek Sparse Attention, achieving an average speedup of 34.93x.
• Autonomous Harness Generation: Can automatically build validation and benchmark scripts for any Git repository based on natural language tasks.
• Robust Validation: Focuses on a strict verification pipeline to avoid measurement noise and "fake" improvements.
• Flexible Execution: Supports local GPU execution or cloud-based execution via Modal.