NVIDIA/cuda-tile

NVIDIA / cuda‑tile – CUDA Tile IR

What it is

• An open‑source compiler infrastructure built on top of MLIR/LLVM that introduces a new CUDA Tile dialect.
• The dialect expresses tile‑based computation patterns that map efficiently to NVIDIA GPUs, especially the Tensor‑Core units.
• It ships with:
  • the MLIR dialect (operations & types for tiled tensors, pointers, etc.)
  • Python bindings for constructing and transforming the IR programmatically
  • a binary bytecode format and tools to translate between MLIR text, bytecode, and final GPU binaries (cubin)
  • a conformance test suite.

Why it matters

• Tiling is a core optimization for high‑performance kernels (matrix‑multiply, convolutions, etc.). By providing first‑class abstractions, CUDA Tile lets developers write kernels at a higher level while still exposing the low‑level control needed to hit the performance of hand‑tuned CUDA.
• Integrated with the CUDA Toolkit 13.1 release, so the generated code can be launched with the regular CUDA driver API.

Key components

Component	Purpose
CUDA Tile Dialect	MLIR dialect that defines tile<…> types, operations like load_ptr_tko, print_tko, reshape, broadcast, etc., tailored for GPU memory hierarchy and Tensor‑Core usage.
Python bindings	Allows scripts to build, inspect, and run transformation passes on Tile IR without writing C++.
Bytecode	Compact binary representation (.tilebc) that can be JIT‑compiled by the CUDA driver or ahead‑of‑time compiled to a cubin with tileiras.
Test suite	LLVM‑lit based tests that verify dialect semantics and the end‑to‑end compilation flow.

Typical workflow

1. Write a kernel in MLIR using the CUDA Tile dialect (see the README example).
2. Translate it to bytecode with cuda-tile-translate.
3. Compile the bytecode to a cubin (tileiras) or load the bytecode directly at runtime – the driver JIT‑compiles it.
4. Launch the kernel from a host program via the CUDA driver API (or runtime API).

Build & installation

• Requires CMake ≥ 3.20, a C++17 compiler, Python ≥ 3.6, and a compatible LLVM/MLIR checkout (the exact commit is pinned in cmake/IncludeLLVM.cmake).
• Three ways to provide LLVM:
  1. Let CMake download the correct LLVM sources automatically (slow but simplest).
  2. Point CMake at a local LLVM source tree.
  3. Link against a pre‑built LLVM installation (must have been built with -DMLIR_ENABLE_BINDINGS_PYTHON=ON if you need the Python API).
• Optional flags:
  • -DCUDA_TILE_ENABLE_BINDINGS_PYTHON=ON – builds the Python API.
  • -DCUDA_TILE_ENABLE_TESTING=ON – builds and runs the lit test suite.
  • -DCUDA_TILE_ENABLE_CCACHE=ON – speeds up recompilation.

Example (from the README)

# Build the library and tools
cmake -G Ninja -S . -B build \
  -DCMAKE_BUILD_TYPE=Release \
  -DCUDA_TILE_ENABLE_BINDINGS_PYTHON=ON \
  -DCUDA_TILE_ENABLE_TESTING=ON
cmake --build build

# Translate an MLIR file to bytecode
cuda-tile-translate example.mlir \
  --bytecode-version=13.1 \
  --mlir-to-cudatilebc --no-implicit-module -o example.tilebc

# Ahead‑of‑time compile to a cubin (replace sm_100 with your GPU arch)
 tileiras --gpu-name sm_100 example.tilebc -o example.cubin

# Compile and run a host program that loads the cubin
 g++ example_host.cpp -o example -I/usr/local/cuda/include \
   -L/usr/local/cuda/lib64 -lcuda
 ./example

The program prints the data array, demonstrating a full end‑to‑end path from Tile IR source to GPU execution.

Use cases

• Researchers building custom kernels for deep‑learning operators that need to exploit Tensor‑Cores.
• Performance engineers who want a reusable, high‑level representation of tiled algorithms while retaining the ability to emit low‑level CUDA.
• Tooling developers extending MLIR with GPU‑specific passes; CUDA Tile provides a concrete dialect and test harness.

Versioning & compatibility

• The open‑source version mirrors the CUDA Toolkit version (13.1.x). The patch number tracks releases of the open‑source code itself.
• Each release is tied to a specific LLVM commit range; a helper script (scripts/get-cuda-tile-version-for-llvm-hash.sh) helps pick the right CUDA Tile tag for a given LLVM revision.

Contribution model

• The project is open‑source under the Apache‑2.0 with LLVM exceptions license, but the README states that external contributions are not accepted at this time. Users are encouraged to file issues for bugs or feedback.

---

Bottom line: CUDA Tile is a genuine, production‑grade compiler stack that lets you write and optimise tiled GPU kernels via an MLIR dialect, with full tooling for translation, testing, and integration into existing CUDA workflows.