pranjalssh/fast.cu

Fastest kernels written from scratch

What it solves

This project provides highly optimized GPU kernels written from scratch to maximize performance on modern NVIDIA hardware, specifically targeting matrix multiplication and sum reduction operations that are critical for AI workloads.

How it works

It implements custom CUDA kernels for specific hardware architectures (H100 and GB300). It focuses on maximizing TFLOPs and memory bandwidth, providing implementations for BF16 matrix multiplication on H100 and NVFP4 matrix multiplication on GB300, as well as a sum reduction kernel for H100.

Who it’s for

Developers and researchers specializing in GPU programming, CUDA optimization, and high-performance computing (HPC) for AI inference and training.

Highlights

  • High-performance BF16 matrix multiplication on H100 that outperforms cuBLAS in some benchmarks.
  • Support for NVFP4 matrix multiplication on GB300/B300 hardware using CUDA 13.1.
  • A sum reduction kernel for H100 that exceeds the performance of the cuBLAS/cub library.

Related

  • Project
    Dao-AILab/quack

    A collection of high-performance GPU kernels for deep learning operations like RMSNorm and Softmax, implemented using CuTe-DSL for modern NVIDIA GPUs.

  • Project
    facebookresearch/xformers

    A toolbox of optimized, customizable building blocks for Transformer models that accelerates research by providing memory-efficient and high-performance CUDA kernels.

  • Project
    MoonshotAI/FlashKDA

    A high-performance CUDA kernel implementation of Kimi Delta Attention (KDA) built on CUTLASS for NVIDIA SM90+ GPUs, serving as an optimized backend for the flash-linear-attention library.

  • Project
    Tencent/hpc-ops

    A production-grade operator library for LLM inference that provides highly optimized CUDA kernels for attention, MoE, and sampling to reduce serving latency.

  • Project
    NVIDIA/cutlass

    A collection of CUDA C++ and Python DSL abstractions for implementing high-performance matrix-matrix multiplication (GEMM) and the related linear algebra operations on NVIDIA GPUs.