warpfront/hipfire

RDNA-native LLM inference engine in Rust.

What it solves

hipfire is a high-performance LLM inference engine specifically optimized for AMD RDNA GPUs (RDNA1 through RDNA4), including consumer cards, pro cards, and APUs. It addresses the difficulty of running LLMs on consumer AMD hardware, where official ROCm support is often limited to datacenter cards, making existing tools like llama.cpp + ROCm often painful to use.

How it works

Built with Rust and HIP, hipfire provides a single binary that avoids Python and PyTorch in the execution path. It uses pre-compiled kernel blobs and JIT-compilation via HIP to target various RDNA architectures. The engine supports multi-GPU expert-parallel serving for large MoE models like DeepSeek V4 Flash, and implements speculative decoding (DFlash) to increase token generation speed.

Who it’s for

Users with AMD RDNA GPUs who want a streamlined, "Ollama-style" experience for running local LLMs with maximum hardware utilization and performance.

Highlights

  • Broad AMD Support: Targets the entire RDNA family (RDNA1 to RDNA4).
  • OpenAI-Compatible API: Includes a background daemon for serving models via an HTTP API.
  • Speculative Decoding: Implements DFlash to significantly boost decode speeds for specific prompts.
  • Multi-GPU Support: Supports expert-parallel sharding across multiple GPUs for large models.
  • Quantization Tools: Includes hipfire quantize for converting HF, safetensors, and GGUF models.
  • Memory Management: Uses CASK-based KV cache eviction to handle long-context prompts without running out of memory.