NVIDIA/TileGym

What it solves

TileGym is a library designed to help developers learn and implement efficient GPU kernels using tile-based programming. It bridges the gap between theoretical GPU optimization and practical application by providing a playground for experimenting with CUDA Tile and integrating these optimized kernels into real-world large language models (LLMs).

How it works

TileGym provides a collection of pre-implemented kernels for common deep learning operators across multiple backends, including cuTile (Python), CUDA Tile C++, Triton CUDA Tile IR, and an experimental Rust-based backend (cuTile-rs). It allows users to swap backends at runtime to compare performance and efficiency. The library includes tools for performance benchmarking and end-to-end integration examples for models like Llama 3.1 and DeepSeek V2.

Who it’s for

It is intended for GPU kernel developers, AI researchers, and engineers looking to optimize LLM implementations or learn the specifics of tile-based GPU programming on NVIDIA Blackwell and Ampere architectures.

Highlights

• Multi-backend support: Supports cuTile, C++, Triton, and Rust for kernel implementation.
• LLM Integration: Provides practical examples of accelerating Llama 3.1 and DeepSeek V2.
• Performance Tooling: Includes micro-benchmarks to evaluate kernel efficiency.
• Hardware Optimized: Specifically validated for NVIDIA Blackwell and Ampere GPUs.