anurmatov/mac-studio-server

Optimized LLM server configuration for Mac Studio and other Apple Silicon Macs. Headless setup with automatic startup, resource optimization, and remote management via SSH.

What it solves

This project transforms an Apple silicon Mac into a headless model server. It simplifies the deployment of LLM backends like Ollama, llama.cpp, DwarfStar (ds4), and MLX-Serve, allowing them to run as services that start automatically at boot.

How it works

The tool provides an automated installer that configures the necessary software (Homebrew, Ollama, llama.cpp, etc.) and manages the system settings. It allows users to select which backends to install and which one should be active, while others remain on standby. It includes a memory guard to manage GPU memory allocation and configures a macOS firewall (pf) to allow secure LAN access to the model servers.

Who it’s for

Users with Apple silicon Macs who want to run a dedicated, headless AI model server for their local network, without needing to maintain manual configurations for each backend.

Highlights

  • Multi-backend support: Supports Ollama, llama.cpp, DwarfStar, and MLX-Serve.
  • Headless optimization: Includes tweaks for macOS to disable sleep, Spotlight, and auto-updates to ensure the server remains available.
  • LAN Access: Integrated firewall allowlists for secure remote access to the model servers.
  • GPU Memory Management: Ability to set a specific percentage of RAM for Metal to use for each backend.
  • Docker Integration: Optional installation and auto-start of Colima and Docker CLI for containerized workloads.

Related

  • Project
    ddalcu/mlx-serve

    A native Zig-based inference server for Apple Silicon that supports MLX and GGUF models with OpenAI, Anthropic, and Ollama API compatibility.

  • Project
    jjang-ai/vmlx

    A self-hosted inference server for Apple Silicon that provides OpenAI, Anthropic, and Ollama compatible APIs for running LLMs, VLMs, and image generation models locally.

  • Project
    containers/ramalama

    An open-source tool that simplifies local AI model inference by packaging models and their dependencies into OCI containers, eliminating host system configuration complexity.

  • Project
    Osmantic/ODS

    A local AI server stack that provides a one-command installation of inference engines, chat interfaces, and automation tools for private, self-hosted AI.

  • Project
    off-grid-ai/OGAM

    A complete offline AI suite for Android, iOS, and macOS that provides text, image, and vision AI capabilities entirely on-device for total privacy.