mudler/LocalAI

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

What it solves

LocalAI acts as a universal, privacy-first AI engine that allows users to run various AI models (LLMs, image, audio, and video generation) on their own hardware without relying ...

How it works

LocalAI uses a modular, composable architecture where a small core manages separate backends. Each backend wraps a specific engine (such as llama.cpp, vLLM, or whisper.cpp) and is pulled as an OCI image only when a model requires it. This ensures that users only install the components they actually use. It provides a unified API that is drop-in compatible with OpenAI, Anthropic, and ElevenLabs, allowing existing applications to switch to local hosting easily.

Who it’s for

It is designed for developers and organizations that prioritize data privacy, want to avoid cloud API costs, or need to run AI capabilities on diverse hardware including NVIDIA, AMD, Intel, and Apple Silicon.

Highlights

  • Multi-modal support: Handles text, vision, voice, image, and video models behind a single API.
  • Hardware flexibility: Runs on everything from high-end GPUs to CPU-only systems.
  • API Compatibility: Drop-in replacement for OpenAI, Anthropic, and ElevenLabs APIs.
  • Agentic capabilities: Includes built-in autonomous agents with tool use, RAG, and Model Context Protocol (MCP) support.
  • Distributed mode: Supports horizontal scaling via PostgreSQL and NATS for production-ready clusters.
  • Modular backends: Over 60 backends available, including native C++/GGML engines for high efficiency.