mozilla-ai/any-llm

Communicate with an LLM provider using a single interface

What it solves

It addresses the fragmented landscape of LLM provider interfaces. Because different providers use slightly different parameter names and response formats, developers often have to rewrite code when switching models. any-llm provides a single, unified interface to access multiple LLM providers without needing to change the core application logic.

How it works

The library acts as a light wrapper that leverages official provider SDKs to ensure maximum compatibility. It offers two primary ways to interact with models:

  1. Direct API Functions: A stateless completion function ideal for scripts and notebooks.
  2. AnyLLM Class: A stateful approach that reuses clients and connection pooling, recommended for production applications.

It supports a provider:model string format or separate parameters to route requests to providers like OpenAI, Anthropic, Mistral, and Ollama, and it can also connect to custom OpenAI-compatible endpoints.

Who it’s for

It is designed for developers who want to switch between different LLM providers easily, those migrating from LiteLLM, and teams building framework-agnostic AI applications that need to avoid dependency on hosted proxy servers.

Highlights

  • Unified Interface: Switch providers by changing a single string.
  • Official SDK Integration: Uses official SDKs rather than reimplementing interfaces to maintain better compatibility.
  • Proxy-free: Operates directly between the code and the provider without requiring a hosted proxy.
  • Developer Experience: Includes full type hints for IDE support and clear error messages.

Related

  • Project
    theopenco/llmgateway

    LLM Gateway is an open‑source API gateway that provides a single OpenAI‑compatible endpoint for routing requests to multiple LLM providers, centralising API‑key management and offering built‑in usage and performance analytics. It can be used as a hosted SaaS service or self‑hosted via Docker.

  • Project
    fruitbars/simple-one-api

    simple‑one‑api is an open‑source Go gateway that unifies multiple LLM providers behind OpenAI‑compatible endpoints, adds capacity‑aware key scheduling, four‑layer rate‑limiting, a built‑in React chat UI, a visual config console, and a Wails desktop client. It can be run as a single binary, Docker container, or desktop app, and stores its configuration in SQLite.

  • Project
    BerriAI/litellm

    An open-source AI gateway and Python SDK that provides a unified OpenAI-compatible interface for calling 100+ different LLM providers.

  • Project
    huggingface/AnyLanguageModel

    AnyLanguageModel is a Swift package that replaces Apple’s FoundationModels import with a unified API for many LLM providers (Apple, Core ML, MLX, llama.cpp, Ollama, OpenAI, Anthropic, Gemini, etc.). It offers a `LanguageModelSession` for prompting, guided generation of Swift structs via `@Generable/@Guide`, tool‑calling support with delegate hooks, optional image inputs, and trait‑based dependency selection to keep binaries small. Installation is via SwiftPM; enable back‑ends with traits (`CoreML`, `MLX`, `Llama`). Security guidance warns against hard‑coding API keys, recommending either user‑provided keys stored in Keychain or a proxy server. The library targets Swift 6.1+, iOS 17/macOS 14+, and works on‑device as well as with cloud APIs.

  • Project
    andrewyng/aisuite

    A lightweight Python library providing a unified Chat Completions API and an Agents API to build AI agents across multiple LLM providers with minimal code.