Arthur-Ficial/apfel

apfel – On‑device LLM for Apple Silicon

What it is – apfel is a Swift‑based command‑line tool that wraps the Apple‑provided Foundation Models LLM that ships on Apple‑silicon Macs. It lets you query the model from the terminal or run a local OpenAI‑compatible HTTP server, so existing SDKs (Python, Node, etc.) can talk to the on‑device model without any cloud keys.

Key features

• Pure on‑device inference – uses the built‑in SystemLanguageModel; no API keys, no network traffic.
• Two usage modes – a pipe‑friendly CLI (apfel "prompt", apfel -f file …) and a drop‑in OpenAI server (apfel --serve listening on http://localhost:11434/v1).
• Tool calling / MCP – can invoke local or remote Model Context Protocol (MCP) tools, with built‑in calculator example.
• Rich output options – plain text, JSON (-o json), schema‑validated JSON, streaming, exit‑code signalling, and file attachment handling (text, PDF, images with OCR/vision on macOS 27).
• Chat REPL – apfel --chat for interactive sessions.
• Config‑less – all behaviour is controlled by flags or environment variables; an optional wrapper apfel‑run adds TOML config support.

Typical workflow

1. Install – brew install apfel (or build from source with Swift 6.3).
2. Run a prompt – apfel "Summarise this repo" or pipe a file: cat README.md | apfel.
3. Use in scripts – JSON output for automation: apfel -o json "Translate to German: hello" | jq .content.
4. Serve locally – apfel --serve and point any OpenAI client at http://localhost:11434/v1.
5. Add tools – apfel --mcp ./mcp/calculator/server.py "What is 15×27?".

Limitations

• Only runs on Apple‑silicon Macs with macOS 26 (Tahoe) or newer; the model size is fixed (~3 B parameters).
• Context window is 4096 tokens on macOS 26 / M1‑M2, 8192 on macOS 27 with M3 + 12 GB.
• No embeddings, no large‑scale parallel speed‑ups – responses take a few seconds.
• Guardrails may block some prompts; --permissive relaxes them.
• Vision support (image understanding) is available only on macOS 27.

Who it’s for

• Developers who want a fast, private LLM for local scripts, CI pipelines, or prototyping without paying for cloud usage.
• Teams that already use OpenAI‑compatible SDKs and would like a drop‑in local backend on Mac workstations.
• Power‑users who enjoy shell‑centric AI assistants and want tool‑calling via MCP.

Where to learn more

• Full install guide: docs/install.md.
• CLI flag reference: docs/cli‑reference.md.
• OpenAI compatibility details: docs/openai‑api‑compatibility.md.
• MCP tool examples: docs/mcp‑calculator.md and the bundled demo/ folder.
• Example prompts: docs/EXAMPLES.md.

Bottom line – apfel turns the Apple‑provided on‑device LLM into a versatile UNIX utility and a local OpenAI‑compatible server, enabling privacy‑first AI workflows on macOS without external APIs.