mudler/LocalAI

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

What it solves

LocalAI provides a unified, privacy-first AI engine that allows users to run a wide variety of AI models (text, vision, audio, image, and video) on their own infrastructure. It eliminates the dependency on cloud-based AI providers by offering a drop-in API compatibility with OpenAI, Anthropic, and ElevenLabs, making it easy to migrate local models to existing applications.

How it works

LocalAI uses a modular, composable architecture where a small core manages various backends. Each backend wraps a best-in-class engine (such as llama.cpp, vLLM, or whisper.cpp) and is delivered as a separate image that is pulled only when a specific model needs it. This ensures that users only install the components necessary for their specific hardware and models. It supports a broad range of hardware acceleration, including NVIDIA, AMD, Intel, Apple Silicon, and Vulkan, or can run on CPU-only systems.

Who it’s for

It is designed for developers and organizations that require high privacy, want to avoid cloud API costs, and need a flexible environment to run multimodal AI models across diverse hardware setups.

Highlights

  • Multimodal Support: Handles LLMs, vision, voice, image, and video models behind a single API.
  • API Compatibility: Drop-in replacement for OpenAI, Anthropic, and ElevenLabs APIs.
  • Modular Design: Backends are pulled on-demand via OCI images to keep the installation lightweight.
  • Hardware Agnostic: Compatible with NVIDIA, AMD, Intel, Apple Silicon, and CPU-only hardware.
  • Agentic Capabilities: Built-in autonomous agents with tool use, RAG, and Model Context Protocol (MCP) support.
  • Enterprise Ready: Includes API key authentication, user quotas, and role-based access control.

Related

  • Project
    cubist38/mlx-openai-server

    An OpenAI-compatible API server for running local MLX models on Apple Silicon, supporting text, multimodal, image, and audio models.

  • Project
    ParisNeo/lollms-webui

    A unified local web interface for accessing hundreds of LLMs and multimodal AI models for text, image, video, and music generation.

  • Project
    open-webui/open-webui

    A self-hosted, extensible AI platform that provides a unified interface for local and cloud-based models, featuring built-in RAG, agent creation, and enterprise-grade user management.

  • Project
    raullenchai/Rapid-MLX

    Rapid‑MLX is a native Apple‑silicon inference engine that runs LLMs, diffusion, audio and video models locally and exposes a drop‑in OpenAI/Anthropic HTTP API, letting any client (LangChain, Claude Code, Aider, etc.) run fully offline on an M‑series Mac.

  • Project
    mlc-ai/mlc-llm

    MLC LLM is a universal deployment engine and ML compiler that enables high-performance native execution of large language models across diverse hardware and platforms.