ardanlabs/kronk

What is Kronk?

Kronk is a Go‑language SDK and a local model‑server that lets developers run AI models on their own machines without needing Python or a separate serving stack. It talks to popular native inference engines (llama.cpp, whisper.cpp, stable‑diffusion.cpp, etc.) and then exposes OpenAI‑compatible HTTP APIs (chat, embeddings, audio transcription, reranking, decision‑making) as well as an Anthropic‑compatible messages endpoint. In short, it gives Go programs and HTTP clients a way to do text generation, vision, embeddings, audio‑to‑text and even experimental image generation locally, with hardware acceleration when available.

Who might use it?

• Go developers who want to embed LLM or multimodal inference directly in their services without pulling in Python runtimes.
• Teams building private‑by‑design AI products that need to keep models and data on‑premise.
• Experimenters who want a quick local server that mimics OpenAI/Anthropic APIs for prototyping RAG, agents, or decision‑making workflows.
• Ops folks who prefer a single binary that can be containerised, managed with standard Go tooling, and still leverages GPU/Metal/Vulkan acceleration.

Key features (from the README)

Feature	Details
Unified Go SDK	Functions for text generation, streaming, tool calls, vision, embeddings, reranking, decision‑model queries, concurrency, and incremental caching.
Model server	HTTP endpoints compatible with OpenAI (/v1/chat/completions, /v1/embeddings, etc.) and Anthropic messages API; also custom /v1/systemone and /v1/decide for decision models.
Hardware acceleration	Uses native libraries: llama.cpp (CPU/GPU), yzma, whisper.cpp, stable‑diffusion.cpp. Supports CUDA, Vulkan, ROCm, Metal, and CPU‑only paths across Linux, macOS, Windows.
Bucky	Speech‑to‑text module built on whisper.cpp, offering transcription, translation, diarisation, and live streaming.
Malina (experimental)	Image‑generation SDK on stable‑diffusion.cpp with ControlNet, ADetailer, AnimateDiff, upscaling, and video generation. Not yet part of the server.
Browser UI	Local web UI (http://localhost:11435) for model management, testing, and monitoring.
Observability & security	Built‑in authentication, rate‑limiting, metrics, tracing, and integration hooks for OpenWebUI, OpenCode, Claude Code.
Easy installation	Homebrew formula, go install CLI, or Docker container; automatic download of compatible native libraries and model files.
Cross‑platform	Linux (amd64/arm64), macOS (amd64/arm64), Windows (amd64/arm64) with appropriate GPU back‑ends.

How to get started (quick‑start steps)

1. Install – on macOS/Linux run brew install ardanlabs/kronk/kronk (or go install github.com/ardanlabs/kronk/cmd/kronk@latest).
2. Start the server – kronk server start.
3. Open the UI – navigate to http://localhost:11435 to upload/download models and test endpoints.
4. Run an example – e.g., make example-question to ask a local LLM a question, or make example-bucky to transcribe audio.
5. Integrate – use the Go SDK (import "github.com/ardanlabs/kronk") or call the OpenAI‑compatible REST API from any client.

How it fits into the AI ecosystem

• Local alternative to cloud LLM APIs – provides the same request shape as OpenAI/Anthropic but runs on‑premise, giving full data control and eliminating latency to external services.
• Bridges Go and native inference – Go developers often lack easy access to ggml‑based engines; Kronk wraps those C/C++ libraries behind idiomatic Go functions.
• Modular architecture – separate subsystems (Yzma for text/vision, Bucky for audio, Malina for images) can be used independently or together, mirroring the multi‑modal trend in modern AI.
• Supports RAG & agents – with chat, embeddings, reranking, and decision endpoints, developers can build Retrieval‑Augmented Generation pipelines or autonomous agents entirely in Go.

Maturity & community

• The project follows upstream ggml releases; breaking changes are documented in BREAKING_CHANGES.md.
• Releases are versioned (e.g., 1.32.9) with a table of compatible library pins.
• Active support via GitHub Issues and direct contact with Bill Kennedy (Ardan Labs).
• Frequent workshops and talks listed in the “Travel Schedule” indicate ongoing community engagement.

---

All information above is taken directly from the repository’s README; no external assumptions have been added.