meta-pytorch/tritonbench

Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.

What it solves

TritonBench provides a standardized way to evaluate the performance of Triton and its integration with PyTorch, ensuring that operators are running efficiently on the hardware.

How it works

It is a collection of PyTorch operators that can be run as a benchmark suite. It integrates kernels from various sources, including Flash-Attention, FBGEMM, and Liger-Kernel, to measure the execution speed and performance of specific operations (such as GEMM).

Who it’s for

Developers and researchers working with Triton and PyTorch who need to benchmark the performance of GPU kernels and operators.

Highlights

  • Supports a wide range of operators via integrated submodules.
  • Can be used as a standalone command-line tool or installed as a library for custom scripts.
  • Integrates with multiple backend projects like CUTLASS and Flash-Attention.

Related

  • Project
    flagos-ai/FlagGems

    A high-performance operator library implemented in Triton that enables LLM training and inference to run across diverse AI accelerators using PyTorch APIs.

  • Project
    ekondis/mixbench

    A benchmark tool for evaluating the performance bounds of GPUs and CPUs by testing mixed operational intensity kernels across various precision levels.

  • Project
    pytorch/kineto

    Kineto is a profiling library used in the PyTorch Profiler to provide performance observability and GPU timeline tracing for machine learning workloads.

  • Project
    cfregly/ai-performance-engineering

    A comprehensive collection of code and resources for optimizing AI systems, focusing on GPU tuning, distributed training, and high-throughput inference scaling.

  • Dispatch
    vLLM Triton Attention Backend Deep Dive

    vLLM has implemented a portable Triton-based attention backend that achieves state-of-the-art performance across NVIDIA, AMD, and Intel GPUs using a single source code implementation.