warpfront/hipfire

RDNA-native LLM inference engine in Rust.

What it solves

hipfire is a high-performance, AMD-native LLM inference engine designed to maximize the potential of AMD GPUs (RDNA and CDNA architectures). It eliminates the overhead associated with Python, PyTorch, and CUDA translation layers by providing its own Rust-based runtime, HIP kernels, and quantization formats, offering an "Ollama-style" user experience for AMD hardware users.

How it works

The project uses a custom Rust runtime and dynamically loads ROCm to execute models. Key technical components include:

  • Redline: A dispatch and retained-replay substrate that records kernel graphs and derives resource dependencies to remove launch overhead after a graph is proven safe.
  • Saddle: A substrate layer being developed to provide first-class compute backends for RDNA, CDNA, and potentially XDNA.
  • Custom Quantization: Supports various MQ formats (e.g., MQ4, MQ4R) and HF4 to optimize throughput and quality.
  • Architecture Tuning: Implements architecture-specific kernels (such as WMMA paths for RDNA3/4) with portable fallbacks for older GPUs (Vega/CDNA).

Who it’s for

Users with AMD GPUs (ranging from consumer RDNA1-4 to professional CDNA/MI300X) who want fast, local LLM and image generation inference without the complexity of a Python-heavy stack.

Highlights

  • AMD-Native: Built with Rust + HIP, removing Python from the hot path for maximum efficiency.
  • Broad GPU Support: Optimized for RDNA3/4, with support for RDNA1/2 and Vega/CDNA architectures.
  • OpenAI-Compatible API: The daemon exposes an API on port 11435 for easy integration with existing clients.
  • Curated Model Registry: Includes over 80 curated model entries, including Qwen, DeepSeek, and FLUX for image generation.
  • C-style CLI: Simple commands for pulling, serving, and chatting with models (e.g., hipfire pull, hipfire chat).

Related

  • Project
    pegainfer-project/pegainfer

    PegaInfer is a Rust‑based, CUDA‑accelerated LLM inference server that provides an OpenAI‑compatible API, supports multiple model families (Qwen3, Gemma 4, DeepSeek‑V2‑Lite, etc.), and offers production‑grade performance without any Python runtime.

  • Project
    lucasjinreal/Crane

    Crane is a Rust‑based inference framework (built on candle) that runs LLMs, vision‑language models, OCR, ASR, TTS and music transcription fast on CPU, CUDA, Metal and experimental ROCm. It loads GGUF checkpoints directly, provides a clean Rust API and an OpenAI‑compatible HTTP server, and ships ready‑to‑use demo binaries for chat, image chat, OCR, speech, etc.

  • Project
    guoqingbao/xinfer

    A blazing-fast LLM inference engine written in pure Rust that eliminates Python/PyTorch dependencies to enable high-throughput, production-ready model serving on consumer and professional GPUs.

  • Project
    raullenchai/Rapid-MLX

    Rapid‑MLX is a native Apple‑silicon inference engine that runs LLMs, diffusion, audio and video models locally and exposes a drop‑in OpenAI/Anthropic HTTP API, letting any client (LangChain, Claude Code, Aider, etc.) run fully offline on an M‑series Mac.

  • Project
    xLLM-AI/xllm

    xLLM is an open‑source C++ inference framework that delivers high‑throughput, low‑latency LLM serving on Chinese AI accelerators (Ascend, Cambricon, Moore Threads, etc.). It separates a brpc‑based service layer from a hardware‑optimized engine, includes hybrid KV‑cache management via Mooncake, and provides ready‑to‑run deployment scripts for models such as GLM‑5.3‑Flash and MiniMax‑M3. The project is Apache‑2.0 licensed and backed by JD.com and several Chinese university labs.