lemonade-sdk/llamacpp-rocm

Fresh builds of llama.cpp with AMD ROCm™ 7 acceleration

What it solves

It provides pre-compiled, nightly builds of llama.cpp specifically optimized for AMD GPUs using ROCm™ 7. This removes the need for users to manually compile the software or install the ROCm SDK separately, as the runtime libraries are built-in.

How it works

The project uses an automated GitHub Actions pipeline to compile llama.cpp against the ROCm SDK (TheRock) and HIP API. It targets a wide range of AMD GPU architectures (from RDNA2 to RDNA4 and CDNA1/2), producing binaries for both Windows and Ubuntu.

Who it’s for

Users with AMD GPUs (including Radeon dGPUs, Ryzen AI APUs, and Instinct accelerators) who want to run GGUF models with high-performance GPU acceleration without the complexity of manual setup.

Highlights

  • Built-in ROCm™ 7: No separate SDK installation is required to run the binaries.
  • Broad AMD Support: Targets multiple architectures including gfx1151, gfx1150, gfx120X, gfx110X, gfx103X, gfx90a, and gfx908.
  • Cross-Platform: Provides nightly builds for both Windows and Ubuntu.
  • Automated Pipeline: Ensures users have access to the freshest cutting-edge builds of the inference engine.

Related

  • Project
    charlie12345/ROCmFPX

    An extension for llama.cpp that introduces AMD-optimized GGUF quantization formats and accelerated kernels to improve LLM inference speed and memory efficiency on Radeon GPUs.

  • Dispatch
    llama.cpp and llama.app: Local LLM Inference and Ecosystem

    llama.cpp and its official home llama.app provide high-performance, hardware-agnostic local LLM inference, now featuring a simplified 'llama serve' command and integration with local coding agents like Pi.

  • Project
    SciSharp/LLamaSharp

    A cross-platform C# library based on llama.cpp that allows developers to run LLaMA and other LLMs locally on CPU and GPU with high-level APIs.

  • Project
    go-skynet/go-llama.cpp

    Go bindings for llama.cpp that enable high-performance local LLM inference within Go applications using the GGUF format.

  • Project
    abetlen/llama-cpp-python

    Python bindings for llama.cpp that enable local LLM inference with an OpenAI-compatible API and support for multiple hardware acceleration backends.