NVIDIA/cutlass

What is CUTLASS?

CUTLASS (CUDA Templates for Linear Algebra Subroutines and Solvers) is a header‑only C++ library that provides building blocks for writing very fast matrix‑matrix multiplication (GEMM) and related linear‑algebra kernels on NVIDIA GPUs. It abstracts the low‑level details of tiling, data movement, and Tensor‑Core instructions so developers can compose high‑performance kernels without hand‑crafting every PTX instruction.

Why it matters for AI/ML

Modern deep‑learning models spend most of their time in GEMM‑style operations (e.g., feed‑forward layers, attention, convolutions). CUTLASS lets framework developers and researchers:

• Target any NVIDIA GPU from Volta to the latest Hopper/Blackwell chips.
• Mix precisions (FP64, FP32, TF32, FP16, BF16, FP8, 8‑bit floating‑point, 4‑bit integer, binary) to trade accuracy for speed.
• Leverage Tensor Cores via the mma/wgmma instructions without writing assembly.
• Plug into existing frameworks – the README lists successful integration tests with FlashAttention, FlashInfer, cuDNN‑Frontend, and PyTorch.

Key components

Component	Role
CuTe DSL	A Python‑native domain‑specific language that generates the same high‑performance CUDA kernels as the C++ templates, dramatically reducing compile time and lowering the C++ expertise barrier.
Primitives API	Low‑level wrappers around NVVM operations, enabling Tensor‑Core programming via SIMT when the higher‑level CuTe abstractions are too restrictive.
Task Scheduling framework	Static analysis of warp‑level schedules; catches concurrency hazards at compile time and visualises resource dependencies.
Operator API (epilogue fusions)	Allows custom post‑processing (e.g., bias addition, activation) to be fused into the GEMM, reducing memory traffic.
Examples & Profiler	Ready‑to‑run demos (e.g., multi‑head attention, backward kernels) and a command‑line profiler for measuring throughput.

How you would use it

1. Add the include/ directory to your project's include path – no library to link because everything is header‑only.
2. Select a GEMM template (e.g., cutlass::gemm::device::Gemm) and specify data types, tile sizes, and layout.
3. Compile with CMake, passing the desired GPU architectures via -DCUTLASS_NVCC_ARCHS="90a" for Hopper or "100a" for Blackwell.
4. Run the provided unit tests (make test_unit -j) to verify the build.
5. For rapid prototyping, write a kernel in Python using the CuTe DSL, which internally generates the same optimized CUDA code.

Who should care?

• Framework engineers (PyTorch, TensorFlow, JAX) who need a reliable source of hand‑tuned GEMM kernels.
• Research scientists experimenting with new precisions (FP8, INT4, binary) that are not yet mainstream in libraries.
• Performance engineers who want fine‑grained control over tiling, memory movement, and kernel scheduling.
• Students learning GPU programming – the DSL and extensive documentation make the learning curve gentler.

Current status (as of CUTLASS 4.7.0 – Aug 2026)

• Supports CUDA 12.8 (optimal) and earlier 11.x toolkits.
• Works on Volta through Blackwell GPUs; Windows builds are still pending a fix.
• CuTe DSL is in public beta and slated to graduate later in 2026.
• New features include the Primitives API, a task‑scheduling analyzer, and improved compiler diagnostics.

---

Bottom line: CUTLASS is a genuine, production‑grade software project that provides the core linear‑algebra primitives powering many state‑of‑the‑art AI workloads on NVIDIA GPUs. It is not a tutorial collection or a simple demo; it is a full‑featured library and DSL ecosystem for high‑performance GPU kernel development.