NVIDIA/cutlass

CUDA Templates and Python DSLs for High-Performance Linear Algebra

What it solves

CUTLASS provides a set of modular abstractions for implementing high-performance matrix-matrix multiplication (GEMM) and related linear algebra computations on NVIDIA GPUs. It solves the complexity of writing highly optimized CUDA kernels by decomposing the parallelization hierarchy and data movement into reusable software components, allowing developers to achieve near-optimal theoretical throughput without writing everything from scratch.

How it works

CUTLASS uses C++ template abstractions and a core library called CuTe to manage hierarchically multidimensional layouts of threads and data. It allows for specialization and tuning via custom tiling sizes, data types, and algorithmic policies. Recently, it introduced CUTLASS DSLs (starting with CuTe DSL), which provide Python-native interfaces for writing high-performance CUDA kernels, reducing compile times and the need for deep C++ expertise while maintaining performance.

Who it’s for

It is designed for performance engineers, researchers, and students who need to implement optimized linear algebra operations on NVIDIA GPUs, ranging from those using low-level C++ to those preferring a Python-based prototyping workflow.

Highlights

  • Broad Hardware Support: Targets NVIDIA architectures from Volta through Blackwell and Rubin.
  • Extensive Data Type Support: Supports a wide range of precisions, including FP64, FP32, TF32, FP16, BF16, FP8, FP4, and narrow integer types.
  • CuTe Library: A specialized system for describing and manipulating tensors of threads and data via layouts and tensors.
  • Python DSL: A native Python interface for writing CUDA kernels that integrates with DL frameworks and speeds up the development cycle.

Related

  • Project
    CNugteren/CLBlast

    A tuned OpenCL BLAS library that provides high-performance linear algebra routines for a wide variety of GPUs and accelerators.

  • Project
    Dao-AILab/quack

    A collection of high-performance GPU kernels for deep learning operations like RMSNorm and Softmax, implemented using CuTe-DSL for modern NVIDIA GPUs.

  • Project
    NVlabs/CuTe

    A pure-Python reference implementation of the CuTe layout-and-tensor algebra used in CUTLASS 3.x, enabling layout prototyping and testing without a GPU.

  • Project
    MoonshotAI/FlashKDA

    A high-performance CUDA kernel implementation of Kimi Delta Attention (KDA) built on CUTLASS for NVIDIA SM90+ GPUs, serving as an optimized backend for the flash-linear-attention library.

  • Project
    NVIDIA/MatX

    MatX is a header‑only C++20 library that provides NumPy‑style tensor operations with lazy expression graphs. By swapping executors you can run the same code on NVIDIA GPUs, on multithreaded CPUs, or as a single JIT‑fused kernel, achieving near‑hand‑written CUDA performance with far less code.