mjun0812/flash-attention-prebuild-wheels

What it solves

Building the flash-attention library from source is a resource-intensive process that takes a very long time. This project provides pre-built binary wheels for various combinations of Python, CUDA, and PyTorch versions, allowing users to install the library quickly without needing to compile it themselves.

How it works

The project uses a combination of GitHub-hosted runners, self-hosted runners, and AWS CodeBuild to automate the compilation of flash-attention (including Flash Attention 3) across multiple platforms. It generates a wide matrix of wheels for Linux (x86_64 and ARM64) and Windows, ensuring compatibility with various glibc versions and hardware requirements (such as Hopper SM90 GPUs for Flash Attention 3).

Who it’s for

Developers and researchers working with Large Language Models who need to use Flash Attention for faster, memory-efficient attention mechanisms but want to avoid the lengthy and complex build process.

Highlights

• Extensive Coverage: Provides over 800 pre-built wheels across Linux and Windows.
• Broad Compatibility: Supports various combinations of Python, CUDA, and PyTorch versions, including those not officially distributed.
• Cross-Platform: Available for Linux x86_64, Linux ARM64, and Windows.
• Simplified Installation: Allows for direct installation via pip install using a provided URL.