ggml-org/Llama-macOS

A cosy home for your LLMs.

What it solves

Llama is a macOS menu bar app that simplifies the process of running local Large Language Models (LLMs) on a Mac. It removes the complexity of manual configuration and installation, providing a user-friendly interface for managing and running models locally and privately.

How it works

The app acts as a wrapper around llama.cpp, running a local server at http://localhost:9931/v1. It automatically detects existing llama.cpp installations or installs a prebuilt binary. It manages GGUF models from Hugging Face, recommending models based on the hardware specifications of the user's Mac. Models are loaded into memory only when requested and unloaded when idle to conserve system resources.

Who it’s for

It is designed for macOS users who want to run local LLMs without needing to deep technical knowledge of CLI tools, as well as developers who want a local API endpoint for their coding agents or other chat UIs.

Highlights

  • 100% local: Data remains on the device for complete privacy.
  • Zero configuration: Models are automatically configured with optimal settings for your Mac.
  • Small footprint: The application is a native macOS app weighing only 4 MB.
  • Model recommendations: Built-in suggestions for models that fit the user's hardware.
  • Standard storage: Uses the Hugging Face cache, ensuring compatibility with other tools like llama.cpp.
  • API access: Provides a standard API for connecting external apps and agents.

Related

  • Dispatch
    llama.cpp and llama.app: Local LLM Inference and Ecosystem

    llama.cpp and its official home llama.app provide high-performance, hardware-agnostic local LLM inference, now featuring a simplified 'llama serve' command and integration with local coding agents like Pi.

  • Project
    Vali-98/ChatterUI

    A native mobile frontend for LLMs that allows users to run models on-device using llama.cpp or connect to various commercial and open-source APIs.

  • Project
    mozilla-ai/llamafile

    A framework that bundles LLMs and speech-to-text tools into single-file executables for zero-installation, cross-platform local execution.

  • Project
    hybridgroup/yzma

    A Go library that integrates llama.cpp for local LLM and VLM inference with hardware acceleration, eliminating the need for CGo or external servers.

  • Project
    ngxson/wllama

    A WebAssembly binding for llama.cpp that enables running LLMs, multimodal models, and tool calling directly in the web browser using WebGPU and Wasm.