apfel: a UNIX tool and OpenAI-compatible server for Apple's built-in on-device LLM

What it solves

apfel provides a way to access the built-in Large Language Model (LLM) provided by Apple FoundationModels on Apple Silicon Macs. It transforms this on-device AI into a usable UNIX tool and a local OpenAI-compatible server, removing the need for API keys, cloud dependencies, and internet connectivity.

How it works

The project acts as a bridge to FoundationModels.SystemLanguageModel. It offers three primary interfaces:

1. UNIX Tool: A command-line interface (CLI) that allows users to pipe text, attach files (including PDFs and images via on-device OCR/extraction), and receive answers directly in the terminal.
2. OpenAI-Compatible Server: A local HTTP server (http://localhost:11434/v1) that allows existing OpenAI SDKs and tools to use the on-device Apple model as a backend.
3. Interactive REPL: A chat mode (--chat) for testing prompts and tools.

It also supports the Model Context Protocol (MCP) to allow the LLM to use external tools (like a calculator) via local or remote servers.

Who it’s for

Developers and power users on macOS 26 Tahoe+ with Apple Silicon (M1+) who want 100% private, on-device AI capabilities integrated into their shell scripts, workflows, and local applications.

Highlights

• 100% On-Device: No cloud, no API keys, and complete privacy.
• OpenAI API Parity: Supports /v1/chat/completions with streaming, JSON schema enforcement, and tool calling.
• File Integration: Native support for attaching PDFs and images for text extraction and analysis.
• MCP Support: Integrates with Model Context Protocol servers to extend model capabilities.
• Shell-First Design: Includes a suite of demos (like cmd for natural language to shell commands) and pipe-friendly output.