zolotukhin/zinc

Zig INferenCe Engine — Local LLM inference on AMD GPUs and Apple Silicon

What it solves

ZINC addresses the performance gap in local LLM inference for consumer GPUs, specifically AMD RDNA3/RDNA4 and Intel Arc. It solves the problem where these GPUs have the necessary hardware (bandwidth and VRAM) but lack optimized software stacks, as tools like vLLM require ROCm (which doesn't support these cards) and other engines use generic backends that don't leverage architecture-specific features.

How it works

ZINC is a high-performance inference engine written in Zig that uses hand-tuned shaders for different hardware platforms instead of a generic compatibility layer:

  • AMD GPUs: Uses Vulkan compute with wave64, cooperative matrix, and architecture-aware tiling.
  • Intel Arc: Uses a Linux Vulkan runtime with Arc-aware device detection and tuning.
  • Apple Silicon: Uses native Metal Shading Language (MSL) kernels with simdgroup reductions and zero-copy mmap for model loading.

It operates as a single binary with no dependency on Python or heavy driver stacks like ROCm, loading models in GGUF format and providing an OpenAI-compatible API and a built-in chat UI.

Who it’s for

It is designed for users running local LLMs on AMD RDNA3/RDNA4 GPUs, Intel Arc GPUs, or Apple Silicon who want maximum inference speed (decode and prefill) without complex setup.

Highlights

  • Hardware-Specific Tuning: Outperforms llama.cpp on AMD RDNA4 across decode, prefill, and end-to-end metrics for several models.
  • Zero-Dependency Runtime: Built with Zig; requires no ROCm or Python installation.
  • Broad Compatibility: Supports GGUF models and provides an OpenAI-compatible /v1 API.
  • Integrated Tooling: Includes a managed model catalog (pull, list) and a built-in browser-based chat interface.