eastriverlee/LLM.swift
LLM.swift is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.
What it solves
LLM.swift is a library that simplifies the integration of large language models (LLMs) into Apple ecosystem apps (macOS, iOS, watchOS, tvOS, and visionOS). It provides a high-level, readable Swift interface for running models locally, removing the complexity of managing low-level C++ bindings or manual prompt formatting.
How it works
The library acts as an abstraction layer over llama.cpp, allowing developers to load GGUF-formatted models from local bundles or directly from Hugging Face. It handles conversation history, chat templates (using the model's embedded Jinja engine or manual overrides), and incremental token evaluation to optimize performance on mobile devices.
Who it’s for
Swift developers building AI-powered applications for Apple platforms who want to run LLMs locally on-device without relying on cloud APIs.
Highlights
- Type-Safe Structured Output: Uses a
@Generatablemacro to automatically generate JSON schemas from Swift structs and enums, ensuring the model produces valid, programmatic responses. - Native Tool Use: Supports function calling by allowing models to interact with Swift code via the
Toolprotocol. - Chain of Thought Support: Provides built-in handling for thinking tokens, allowing developers to separate a model's internal reasoning process from its final answer.
- Local Model Management: Supports loading models from local files or downloading them directly from Hugging Face.
- Text Embeddings: Includes built-in support for generating and comparing text embeddings for semantic search and similarity tasks.