NVIDIA/cuda-tile

CUDA Tile IR is an MLIR-based intermediate representation and compiler infrastructure for CUDA kernel optimization, focusing on tile-based computation patterns and optimizations targeting NVIDIA tensor core units.

What it solves

CUDA Tile IR simplifies the development of high-performance CUDA kernels by providing a specialized intermediate representation (IR) and compiler infrastructure. It specifically targets the optimization of tile-based computation patterns and the utilization of NVIDIA tensor core units, reducing the complexity of managing memory hierarchies and GPU-specific optimizations manually.

How it works

Built on the MLIR (Multi-Level Intermediate Representation) framework, the project provides a domain-specific dialect for expressing tiled computations. It includes Python bindings for programmatic IR construction and a bytecode format for efficient serialization. Programs can be compiled from MLIR to bytecode and then either JIT-compiled via the CUDA driver API or compiled ahead-of-time (AoT) into cubin files using the tileiras tool.

Who it’s for

It is designed for developers creating high-performance GPU kernels who want to use higher-level abstractions for tiling and tensor core optimizations rather than writing raw CUDA code.

Highlights

  • MLIR-based Dialect: Provides first-class operations and types specifically for tile-based computations.
  • Python API: Enables programmatic manipulation and transformation of the IR.
  • Flexible Compilation: Supports both Just-In-Time (JIT) and Ahead-of-Time (AoT) compilation paths.
  • Tensor Core Optimization: Specifically engineered to target NVIDIA tensor core units for maximum performance.

Related

  • Project
    buddy-compiler/buddy-mlir

    An MLIR-based compiler framework for co-designing DSLs and DSAs, enabling the optimization and deployment of AI models like DeepSeek R1 and Whisper across various hardware targets.

  • Project
    sophgo/tpu-mlir

    An MLIR-based machine-learning compiler that converts pre-trained neural networks from frameworks like PyTorch and ONNX into optimized bmodel files for TPU execution.

  • Project
    iree-org/iree

    IREE is an MLIR-based compiler and runtime that lowers machine learning models to a unified IR for efficient deployment across data centers, mobile, and edge devices.

  • Project
    NVIDIA/tilus

    Tilus is a tile-level GPU kernel programming language that enables high-performance, low-precision computation with explicit control over shared memory and registers.

  • Project
    NVIDIA/TileGym

    A CUDA Tile kernel library providing tutorials and examples for tile-based GPU programming to optimize deep learning operators and LLM inference.