zolotukhin/zinc

Zig INferenCe Engine — Local LLM inference on AMD GPUs and Apple Silicon

What it solves

ZINC is a high-performance, local inference engine designed to run GGUF models on a wide variety of consumer GPUs. It aims to provide a faster alternative to existing tools like llama.cpp, particularly on AMD Radeon hardware using ROCm.

How it works

Written in Zig, ZINC is a single binary that integrates several components: a command-line interface, a browser-based chat interface, a model manager for downloading and managing GGUF files, and an OpenAI-compatible API server. It supports multiple GPU backends including Vulkan (AMD/Intel), ROCm/HIP (AMD), Metal (Apple Silicon), and CUDA (NVIDIA, experimental).

Who it’s for

Developers and AI enthusiasts who want to run LLMs locally on their own hardware with high performance, especially those using AMD or Apple Silicon GPUs.

Highlights

  • Fast GGUF inference with native kernels for various GPU paths.
  • OpenAI-compatible API for easy integration into other applications.
  • Integrated model manager and browser chat for immediate use.
  • Broad hardware support across AMD, Intel, Apple, and NVIDIA GPUs.

Related

  • Project
    ddalcu/mlx-serve

    A native Zig-based inference server for Apple Silicon that runs LLMs and multimodal models with OpenAI, Anthropic, and Ollama API compatibility.

  • Project
    zml/zml

    A production inference stack that decouples AI workloads from proprietary hardware, allowing a single codebase to run models on NVIDIA, AMD, Intel, and TPU/Trainium accelerators.

  • Project
    lmstudio-ai/mlx-engine

    An Apple MLX-based LLM engine for LM Studio that enables optimized local inference of text and vision models on macOS.

  • Project
    raullenchai/Rapid-MLX

    Rapid‑MLX is a native Apple‑silicon inference engine that runs LLMs, diffusion, audio and video models locally and exposes a drop‑in OpenAI/Anthropic HTTP API, letting any client (LangChain, Claude Code, Aider, etc.) run fully offline on an M‑series Mac.

  • Dispatch
    Optimizing Local Inference: A Deep Dive into the DeepSeek 4 Flash Metal Engine

    An exploration of a specialized local inference engine for DeepSeek 4 Flash on Apple Metal, highlighting the shift toward hardware-specific optimization and the challenges of large-context processing.