harbor: a unified orchestrator for deploying and managing a complete local LLM stack

What it solves

Harbor simplifies the complex process of setting up and managing a local AI stack. Instead of manually configuring multiple Docker containers, environment variables, and network connections between backends, frontends, and supporting services, Harbor provides a unified CLI and companion app to deploy and orchestrate these components effortlessly.

How it works

Harbor acts as an orchestration layer over Docker Compose. It allows users to spin up pre-configured services (like Ollama, vLLM, or Open WebUI) using simple commands like harbor up [service]. It handles the cross-service connectivity and configuration automatically, ensuring that frontends are pre-wired to their respective backends. It also includes a harbor launch command to integrate local backends with host-based coding tools and agents.

Who it’s for

It is designed for developers, AI researchers, and enthusiasts who want to run a complete, integrated local LLM environment—including RAG, voice chat, and image generation—without the overhead of manual infrastructure setup.

Highlights

• One-Command Deployment: Quickly launch backends (Ollama, llama.cpp, vLLM), frontends (Open WebUI), and satellites (SearXNG, Speaches) with harbor up.
• Agentic Coding Integration: The harbor launch command wires local models into host tools like Claude Code, Codex, and VS Code.
• Harbor Boost: A system for chaining agentic modules (web research, audits, scope checks) into custom workflows for coding agents.
• Broad Backend Support: Supports a wide array of inference engines, including host-native Metal/GPU acceleration for macOS via MLX and oMLX.
• Local Web RAG: Pre-connects search engines like SearXNG to various UIs and deep research tools.
• Infrastructure Tools: Includes built-in tunneling for remote access, QR codes for mobile access, and an "eject" feature to export the setup into a standalone Docker Compose file.