go-skynet/go-llama.cpp

LLama.cpp golang bindings

What it solves

This project provides Go (Golang) bindings for llama.cpp, allowing developers to integrate Large Language Model (LLM) inference capabilities directly into Go applications without needing to write C++ code.

How it works

It acts as a high-level wrapper around the llama.cpp library. To maintain performance and avoid the overhead associated with calling C functions from Go (CGO), the project keeps most of the computational heavy lifting in C/C++ code while providing a simplified Go interface for the developer.

Who it’s for

Go developers who want to run LLMs locally using the gguf file format and leverage hardware acceleration.

Highlights

  • Hardware Acceleration: Supports multiple backends including OpenBLAS, CuBLAS (NVIDIA), ROCM (AMD), OpenCL, and Metal (Apple Silicon).
  • GGUF Support: Fully compatible with the new gguf model format.
  • High-Level Interface: Designed to be simple to use while minimizing the computational cost of the CGO bridge.

Related

  • Project
    hybridgroup/yzma

    A Go library that integrates llama.cpp for local LLM and VLM inference with hardware acceleration, eliminating the need for CGo or external servers.

  • Project
    abetlen/llama-cpp-python

    Python bindings for llama.cpp that enable local LLM inference with an OpenAI-compatible API and support for multiple hardware acceleration backends.

  • Project
    ggml-org/llama.cpp

    A high-performance C/C++ implementation for local LLM and VLM inference, supporting a wide range of hardware through quantization and hardware acceleration.

  • Project
    SciSharp/LLamaSharp

    A cross-platform C# library based on llama.cpp that allows developers to run LLaMA and other LLMs locally on CPU and GPU with high-level APIs.

  • Project
    withcatai/node-llama-cpp

    A Node.js library that provides pre-built bindings for llama.cpp, enabling developers to run LLMs locally with automatic hardware acceleration and TypeScript support.