huggingface/AnyLanguageModel

AnyLanguageModel – Swift‑first abstraction over many LLM back‑ends

What it is

• A Swift package that lets you write code against a single, Apple‑style API for language models and then swap the underlying provider (Apple Foundation Models, Core ML, MLX, llama.cpp, Ollama, Anthropic, OpenAI, Gemini, etc.).
• You only change the import line (import FoundationModels → import AnyLanguageModel) and the rest of your code stays the same.

Key concepts

• LanguageModelSession – the object that holds a model instance and an optional list of tools (functions the model can call). All interactions happen through session.respond { … }.
• Guided generation – use the @Generable and @Guide property wrappers to ask the model to produce a strongly‑typed Swift struct instead of parsing raw text.
• Tool calling – define a Tool conforming type (e.g., a weather lookup) and let the model decide when to invoke it. A delegate (ToolExecutionDelegate) can observe or approve each call.
• Traits – Swift‑Package‑Manager traits let you opt‑in only to the heavy back‑ends you need (CoreML, MLX, Llama). This keeps binary size low.

Supported providers (checkboxes in the README indicate they are implemented)

• Apple Foundation Models (system model on iOS 26/macOS 26+)
• Core ML (on‑device .mlmodelc files)
• MLX (Apple‑silicon‑accelerated models via mlx‑swift)
• llama.cpp (GGUF quantised models)
• Ollama HTTP API (local or remote Ollama server)
• Anthropic Messages API
• Google Gemini API
• OpenAI Chat Completions & Responses APIs
• Open Responses (any endpoint compatible with the OpenAI responses format)

Installation

// Package.swift
dependencies: [
    .package(url: "https://github.com/huggingface/AnyLanguageModel", from: "0.11.0")
]

If you need a specific backend, enable its trait:

.package(
    url: "https://github.com/huggingface/AnyLanguageModel",
    from: "0.11.0",
    traits: ["CoreML", "MLX"]
)

When using traits you must also add the underlying packages (CoreML → huggingface/swift‑transformers, MLX → ml‑explore/mlx‑swift‑lm, Llama → mattt/llama.swift). The README provides a full Xcode‑shim workflow for projects that cannot declare traits directly.

Typical usage

import AnyLanguageModel

let model = SystemLanguageModel.default          // or CoreMLLanguageModel(...), MLXLanguageModel(...), etc.
let session = LanguageModelSession(model: model)

// Simple prompt
let resp = try await session.respond { Prompt("Explain quantum computing in one sentence") }
print(resp.content)

Guided generation example

@Generable(description: "Basic profile information about a cat")
struct CatProfile {
    var name: String
    @Guide(description: "The age of the cat", .range(0...20))
    var age: Int
    @Guide(description: "One‑sentence personality description")
    var profile: String
}

let profile = try await session.respond(
    to: "Generate a cute rescue cat",
    generating: CatProfile.self
).content

The model returns a CatProfile instance directly.

Tool calling

struct WeatherTool: Tool {
    let name = "getWeather"
    let description = "Retrieve the latest weather information for a city"

    @Generable
    struct Arguments { @Guide var city: String }

    func call(arguments: Arguments) async throws -> String {
        "The weather in \(arguments.city) is sunny and 72°F"
    }
}

let session = LanguageModelSession(model: model, tools: [WeatherTool()])
let answer = try await session.respond { Prompt("How's the weather in Cupertino?") }
print(answer.content)

A delegate can be attached to watch or approve the tool call.

Image inputs Many cloud providers (OpenAI, Anthropic, Gemini, Open Responses) and some local back‑ends (MLX, Ollama) accept images:

let resp = try await session.respond(
    to: "Describe what you see",
    images: [.init(url: URL(string: "https://example.com/photo.jpg")!)]
)

The table in the README lists which providers support images.

Security guidance The README stresses never hard‑coding API keys. Two production patterns are recommended:

1. Bring‑Your‑Own‑Key – store the user‑provided key in the system Keychain and send it directly to the provider.
2. Proxy server – keep the provider key on a backend you control, expose a short‑lived token to the app, and forward requests. Both approaches are explained with their trade‑offs.

Why you might use it

• Write once, run everywhere: the same Swift code works on‑device (Core ML, MLX, llama.cpp) and in the cloud (OpenAI, Anthropic, Gemini, Ollama).
• Strongly‑typed outputs via guided generation reduce brittle string parsing.
• Built‑in tool‑calling support lets you build agent‑style apps (e.g., assistants that can fetch weather, look up data, or run custom code).
• Trait‑based dependency management keeps the final app lightweight.

Current limitations

• Apple Foundation Models require iOS/macOS 26, which at the time of writing is a future OS version.
• llama.cpp does not support tool calling.
• The LiteRT backend was removed in v0.11 due to build‑time issues.

Where to learn more

• The README links to Apple’s Guided Generation docs, the various provider APIs, and a sample Xcode app (chat‑ui‑swift).
• Issues #15 and #135 discuss known Xcode/SwiftPM bugs and work‑arounds.

---

Bottom line: AnyLanguageModel is a genuine, production‑ready Swift library that abstracts over a wide range of language‑model providers, adds first‑class tool calling and typed generation, and uses Swift‑Package‑Manager traits to keep binaries small. It’s aimed at iOS/macOS/visionOS developers who want a unified API for on‑device and cloud LLMs.