gpu-mode/reference-kernels

Official Problem Sets / Reference Kernels for the GPU MODE Leaderboard!

What it solves

This repository provides a centralized collection of reference implementations for GPU kernel competitions. It ensures that participants in competitions hosted by KernelBot on the GPU Mode community (//discord.gg/gpumode) can have a baseline for correctness and performance evaluation against which their own custom GPU kernels can be measured.

How it works

The project organizes problems into specific directories (e.g., PMPP practice problems, AMD and NVIDIA competitions, BioML kernels, and university course assignments). Each problem contains a a PyTorch reference implementation (reference.py), a problem specification for generating test cases (task.yml), and a schema for inputs and outputs (task.py).

Who it’s for

Developers and researchers specializing in high-performance computing (HPC) and GPU programming, as well as students in parallel programming classes at universities like Stanford and Princeton.

Highlights

  • own reference implementations for various hardware platforms including AMD and NVIDIA GPUs.
  • Supports integration with NVIDIA Nsight Compute for profiling and verification.
  • Provides infrastructure for university parallel programming courses.
  • Includes a variety of problem sets ranging from linear algebra to BioML kernels.

Related

  • Project
    KernelTuner/kernel_tuner

    An auto-tuning tool for GPU kernels that benchmarks and optimizes parameters across multiple GPU programming languages to improve performance and and energy efficiency.

  • Project
    ekondis/mixbench

    A benchmark tool for evaluating the performance bounds of GPUs and CPUs by testing mixed operational intensity kernels across various precision levels.

  • Project
    gau-nernst/learn-cuda

    A collection of CUDA kernel implementations and educational resources for PyTorch users to learn GPU programming and hardware optimization.

  • Project
    meta-pytorch/KernelAgent

    A multi-agent system that converts PyTorch programs into verified Triton kernels and iteratively optimizes their performance using hardware profiling and LLM-guided bottleneck diagnosis.

  • Project
    NVIDIA/TileGym

    A CUDA Tile kernel library providing tutorials and examples for tile-based GPU programming to optimize deep learning operators and LLM inference.