scouzi1966/maclocal-api

What it solves

AFM (Apple Foundation Models) turns an Apple Silicon Mac into a private, OpenAI-compatible AI server. It removes the need for cloud accounts, API keys, and Python runtimes for serving, allowing users to run high-performance local LLMs and multimodal services natively on macOS.

How it works

AFM is a native Swift executable that provides an application layer over AFMKit. It supports multiple runtimes for different model types:

• MLX: For open models, VLMs, and performance tuning.
• Apple Foundation Models: For zero-download system models and LoRA adapters.
• DwarfStar: For compatible fixed-schedule Metal checkpoints.
• Gateway: To aggregate models from other local servers like Ollama or LM Studio.

It exposes an OpenAI-compatible API surface for chat, embeddings, vision OCR, and speech-to-text/text-to-speech, and includes a built-in WebUI and a native terminal-based chat interface (TUI).

Who it’s for

Developers building AI agents or native macOS apps who need local, private inference, as well as users who want to run local LLMs with an OpenAI-compatible interface for existing SDKs and clients.

Highlights

• OpenAI Compatibility: Seamlessly connects with clients like Cursor, Aider, and Continue.
• Agent-Ready: Features native tool-calling formats, structured output (JSON schema), and prefix KV caching for multi-turn loops.
• Apple-Native Integration: Direct access to macOS system frameworks for OCR, speech recognition, and text-to-speech.
• High Performance: Supports speculative decoding (MTP, Eagle3), concurrent requests, and xgrammar for strict schema enforcement.
• Deterministic Evaluation: Includes a built-in, no-judge suite for evaluating local model performance and timing.