zolotukhin/zinc

Zig INferenCe Engine — Local LLM inference on AMD GPUs and Apple Silicon

What it solves

ZINC allows you to run GGUF-format language models locally on your own GPU hardware. It bridges the gap between raw model files and usable applications by providing a single, native binary that handles model execution, chat interfaces, and API hosting.

How it works

Built using the Zig programming language, ZINC acts as an inference engine. It supports multiple GPU backends—including Vulkan, Metal, and ROCm—to ensure compatibility across different hardware like AMD, Apple Silicon, and NVIDIA. Once running, it provides a command-line interface for model management, a browser-based chat UI, and an OpenAI-compatible API server that allows other applications to connect to your local model.

Who it’s for

It is designed for developers and power users who want to run large language models locally on their own hardware, whether for private chat, automated coding tasks, or integrating local AI into existing software workflows.

Highlights

  • Runs as a single native binary for simplified deployment.
  • Includes a built-in browser chat interface and a local API server compatible with OpenAI standards.
  • Supports diverse hardware backends including Vulkan, Metal, ROCm, and CUDA.
  • Features a managed model catalog for easy downloading and testing of popular model families like Qwen and Gemma.
  • Provides integration tools for automated coding harnesses, allowing agents to run tests and perform repository-level tasks.

Related

  • Project
    ashhart/TensorFold

    A native Zig-based inference engine that serves language models on Apple Silicon and NVIDIA GPUs without requiring Python or MLX.

  • Project
    pegainfer-project/pegainfer

    PegaInfer is a Rust‑based, OpenAI‑compatible LLM inference server that runs directly on NVIDIA GPUs using CUDA kernels, offering low‑latency, low‑memory serving without Python or PyTorch.

  • Project
    Michael-A-Kuykendall/shimmy

    Shimmy is a high-performance, single-binary local inference server for GGUF models that provides an OpenAI-compatible API without requiring Python or C++ dependencies.

  • Project
    ddalcu/mlx-serve

    A native Zig-based inference server for Apple Silicon that supports MLX and GGUF models with OpenAI, Anthropic, and Ollama API compatibility.

  • Project
    trymirai/uzu

    uzu is a cross-platform inference engine for running AI models locally in your app, offering zero-latency, private, and cost-free inference with a simple API and support for multiple programming languages.