ardanlabs/kronk

Go With Your Own Intelligence! Use Go for hardware accelerated local inference with llama.cpp, whisper.cpp, and stable-diffusion.cpp directly integrated into your Go applications. Kronk provides a high-level API and production ready model server.

What it solves

Kronk provides a way to run local, hardware-accelerated AI models without needing a Python environment or a complex model-serving stack. It simplifies the process of integrating AI capabilities—like text generation, image creation, and speech-to-text—directly into Go applications or as a standalone server.

How it works

Kronk acts as a high-level Go SDK and model server that wraps native inference engines. It integrates with several specialized libraries:

  • Kronk (Core): Uses llama.cpp and yzma for text, vision, embedding, and reranking models.
  • Bucky: Uses whisper.cpp and bucky for speech-to-text (transcription and translation).
  • Malina: An experimental SDK using stable-diffusion.cpp for image and video generation.

The project provides a hardware-accelerated backend supporting CUDA, Vulkan, HIP, ROCm, SYCL, Metal, and OpenCL across Linux, macOS, and Windows.

Who it’s for

Go developers who want to embed AI inference directly into their binaries or those who need a local, OpenAI- and Anthropic-compatible API server for managing and running multiple AI models.

Highlights

  • Multi-modal support: Handles text, vision, audio transcription, and image/video generation.
  • OpenAI/Anthropic Compatibility: The model server provides APIs compatible with standard industry interfaces.
  • Hardware Acceleration: Native support for a wide range of GPU backends across multiple operating systems.
  • Go-native experience: Offers both an SDK for in-process inference and a full model server with a browser UI for management.
  • Advanced AI features: Supports tool calls, reasoning, incremental message caching, and ControlNet conditioning for images.

Related

  • Project
    replicate/cog

    An open-source tool that packages machine learning models into production-ready Docker containers and automatically generates an HTTP inference server.

  • Project
    tetherto/qvac

    A local AI SDK and model provider that enables running LLMs, speech, vision, and generative AI workloads on any device without cloud dependencies.

  • Project
    Ingenimax/agent-sdk-go

    A powerful Go framework for building production-ready AI agents with integrated memory management, multi-LLM support, and enterprise-grade observability.

  • Project
    eleiton/ollama-intel-arc

    A Docker-based deployment suite that enables Intel Arc GPU acceleration for Ollama, Stable Diffusion, and OpenAI Whisper via a unified interface.

  • Project
    EricLBuehler/mistral.rs

    A fast, flexible LLM inference engine written in Rust that supports multimodal models, agentic tool execution, and OpenAI/Anthropic compatible APIs.