mudler/LocalAI

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

What it solves

LocalAI provides a unified, privacy-first AI engine that allows users to run a wide variety of AI models (LLMs, image, audio, and video generation) on their own hardware without relying on external cloud providers. It eliminates the need for expensive GPUs by supporting CPU-only execution and a broad range of hardware acceleration options.

How it works

LocalAI uses a modular, composable architecture where the core engine is lightweight and specific backends (such as llama.cpp, vLLM, and whisper.cpp) are pulled as separate OCI images only when a model requires them. This ensures users only install the components they actually use. It provides drop-in API compatibility with OpenAI, Anthropic, and ElevenLabs, allowing existing applications to switch to local hosting easily.

Who it’s for

It is designed for developers and organizations that require high privacy, want to avoid cloud API costs, or need to run AI models on diverse hardware including NVIDIA, AMD, Intel, and Apple Silicon.

Highlights

  • Multi-modal support: Handles text, vision, voice, image, and video models behind a single API.
  • Hardware agnostic: Runs on everything from high-end GPUs to CPU-only environments.
  • Agentic capabilities: Includes built-in autonomous agents with RAG, tool use, and Model Context Protocol (MCP) support.
  • Enterprise ready: Features multi-user support with API key authentication, role-based access, and user quotas.
  • Distributed mode: Supports horizontal scaling via PostgreSQL and NATS for production environments.

Related

  • Project
    raullenchai/Rapid-MLX

    A high-performance LLM inference server for Apple Silicon that optimizes MLX for faster local inference and reliable tool-calling for coding agents.

  • Project
    ParisNeo/lollms-webui

    A unified local web interface for accessing hundreds of LLMs and multimodal AI models for text, image, video, and music generation.

  • Project
    cubist38/mlx-openai-server

    An OpenAI-compatible API server for running local MLX models on Apple Silicon, supporting text, multimodal, image, and audio models.

  • Project
    mlc-ai/mlc-llm

    MLC LLM is a universal deployment engine and ML compiler that enables high-performance native execution of large language models across diverse hardware and platforms.

  • Project
    mudler/LocalAGI

    LocalAGI is a self‑hosted Go platform for building and running AI agents locally. It provides a web UI for no‑code agent creation, OpenAI‑compatible REST APIs, built‑in RAG/knowledge base, multimodal model support, and extensible Go‑based custom actions. Agents can be linked into teams, scheduled, and connected to Discord, Slack, Telegram, GitHub, and IRC. Docker‑Compose files let you run on CPU or various GPUs, keeping all data on‑premise for privacy.