NVIDIA/TileGym

Helpful kernel tutorials, examples and SKILLs for tile-based GPU programming

What it solves

TileGym is a library designed to help developers learn and implement efficient tile-based GPU programming using CUDA. It bridges the gap between theoretical GPU optimization and practical application by providing a playground for experimenting with high-performance kernels, specifically targeting deep learning operators and large language models (LLMs).

How it works

TileGym provides a collection of pre-implemented CUDA Tile kernels across multiple backends, allowing users to swap implementations to compare performance and behavior. It supports various backends including cuTile (Python), CUDA Tile C++, Triton CUDA Tile IR, and an experimental Rust-based backend (cuTile-rs).

Who it’s for

It is intended for GPU kernel developers, AI researchers, and software engineers looking to optimize the performance of deep learning models, particularly those working with NVIDIA Blackwell and Ampere architectures.

Highlights

  • Multi-backend support: Offers implementations in cuTile, C++, Triton, and Rust.
  • LLM Integration: Includes end-to-end examples showing how to integrate these kernels into models like Llama 3.1 and DeepSeek V2.
  • Performance Tooling: Built-in benchmarking tools to evaluate kernel efficiency.
  • Hardware Optimized: Specifically validated for NVIDIA Blackwell and Ampere GPUs.

Related

  • Project
    deepseek-ai/DeepGEMM

    A high-performance tensor core kernel library for NVIDIA GPUs that provides optimized GEMM, MoE, and MQA kernels to accelerate LLM training and inference.

  • Dispatch
    Mastering CUDA: Navigating the Landscape of GPU Programming Resources

    An exploration of the best resources for learning CUDA, from foundational books to modern kernel source code and high-level Python frameworks.

  • Project
    NVIDIA/cutile-python

    A Python-based programming language for NVIDIA GPUs that allows developers to write high-performance kernels using Tile IR.

  • Project
    NVIDIA/cuda-tile

    An MLIR-based intermediate representation and compiler infrastructure for optimizing tile-based computations and NVIDIA tensor core usage in CUDA kernels.

  • Project
    NVIDIA/cutlass

    A collection of CUDA C++ and Python DSL abstractions for implementing high-performance matrix-matrix multiplication (GEMM) and the related linear algebra operations on NVIDIA GPUs.