NVlabs/cutile-rs

What it solves

cuTile Rust provides a way to write GPU kernels that are memory-safe and free from data races, using idiomatic Rust. It eliminates the common risks of manual memory management and concurrency bugs in GPU programming by extending Rust's ownership and borrowing rules across the boundary between the host (CPU) and the device (GPU).

How it works

The system uses a tile-based approach to manage GPU memory. Mutable tensors are partitioned into disjoint pieces before they are launched, while immutable tensors are shared. The #[cutile::module] macro captures the Rust AST for a kernel and JIT-compiles it through CUDA Tile IR into a GPU cubin. This allows the system to preserve ownership and access disciplines (such as exclusive mutable access or shared read-only access) directly in the device code.

Who it’s for

Developers who want to write high-performance GPU kernels in Rust without the safety risks associated with traditional CUDA C++ programming, specifically those targeting NVIDIA GPUs with compute capability sm_80 or higher.

Highlights

• Memory Safety: Extends Rust's ownership discipline to GPU kernels to prevent data races.
• JIT Compilation: Uses a macro-based system to compile Rust ASTs into CUDA Tile IR.
• High Performance: Reaches up to 92% of dense f16 peak performance on NVIDIA B200 for GEMM operations, competitive with cuBLAS.
• Flexible Execution: Supports synchronous launches, asynchronous pipelines, and CUDA graph replay.
• Integration: Used as the foundation for Grout, a Qwen 3 inference engine.