RightNow-AI/autokernel

What it solves

Writing fast GPU kernels (the low-level code that runs neural networks) is slow and requires deep expertise. AutoKernel automates this by turning a coding AI agent loose on your model: it finds the slowest parts, extracts them as separate kernels, and keeps trying optimizations until they are faster. Instead of manually tuning each operation, you give it a model and get back optimized kernels with measured speedups.

How it works

You provide any PyTorch model. AutoKernel runs a pipeline:

1. Profile: Uses PyTorch's profiler to rank which GPU kernels take the most time.
2. Extract: Pulls out each bottleneck as a standalone Triton or CUDA C++ kernel file.
3. Optimize: An AI coding agent (like Claude or Codex) reads a set of instructions (program.md), edits one kernel file, runs a fixed benchmark, and keeps the change only if it improves speed while passing correctness checks. It repeats this loop forever, with each experiment taking about 90 seconds.
4. Verify: After optimizing all bottlenecks, it plugs the new kernels back into the model and reports the total end-to-end speedup.

The benchmark includes a 5-stage correctness check (smoke, shape sweep, numerical stability, determinism, edge cases) before measuring performance. An orchestrator uses Amdahl's law to decide which kernel to work on next, prioritizing those with the biggest impact on the whole model.

Who it's for

Developers and researchers who need faster GPU inference or training for PyTorch models, but don't want to hand-write CUDA or Triton kernels. It requires an NVIDIA GPU (tested on H100/A100/RTX 4090) and Python 3.10+. It ships with ready-to-run model definitions (GPT-2, LLaMA, BERT) so you can try it without external model libraries.

Highlights

• Autonomous optimization: Runs hundreds of experiments overnight without supervision, editing one file and keeping only successful changes.
• Dual backend: Generates both Triton (fast iteration) and native CUDA C++ (max performance) kernels, sharing the same interface.
• 9 kernel types: Covers common deep-learning operations like matmul, softmax, layernorm, flash attention, fused MLP, and more.
• KernelBench integration: Works with the standard benchmark for AI-generated GPU kernels, running iterative refinement instead of one-shot guesses.
• Correctness-first: Any kernel that fails the 5-stage correctness tests is immediately reverted, so speedups are always safe.
• Simple output: Every experiment logged to a plain results.tsv file for easy tracking.