ROCm/composable_kernel

What it solves

Composable Kernel (CK) provides a programming model for creating high-performance kernels for machine learning workloads. It addresses the challenge of maintaining performance portability across different hardware architectures (such as various GPUs and CPUs) while keeping the code maintainable.

How it works

CK uses HIP C++ and two primary technical concepts to achieve its goals:

1. Tile-based programming model: Organizes computations into tiles to optimize data movement and processing.
2. Tensor Coordinate Transformation: A technique used to reduce the algorithmic complexity of complex machine learning operators.

The library is organized into four layers: Templated Tile Operators, Templated Kernel and Invoker, Instantiated Kernel and Invoker, and a Client API.

Who it’s for

It is designed for developers writing performance-critical ML kernels who need their code to run efficiently across multiple hardware targets.

Highlights

• Performance Portability: Supports multiple GPU architectures (e.g., gfx908, gfx90a, gfx1100) and CPUs.
• Multi-layer Architecture: Separates templated logic from instantiated kernels for better maintainability.
• Broad Data Type Support: Includes support for various precisions including fp64, fp32, tf32, fp16, fp8, bf16, and int8.
• Integrated Tooling: Includes a profiler (ckProfiler) and a codegen system.