av/harbor

Harbor – One‑command local LLM stack

What it is – Harbor is a CLI + desktop‑style companion app that orchestrates a whole family of AI services (LLM back‑ends, chat front‑ends, web‑search, voice, image generation, workflow tools, etc.) with Docker‑Compose. With a single harbor up … you get a ready‑to‑use stack: the model server (Ollama, llama.cpp, vLLM, …), a UI such as Open WebUI, plus optional satellites like SearXNG for web‑RAG, Speaches for speech‑to‑text / text‑to‑speech, ComfyUI for image generation, and many more.

---

Core ideas

Idea	What it means
Zero‑config start	harbor up pulls the required Docker images, creates the compose file, wires the services together and launches them. No manual networking or env‑file editing needed.
Pluggable back‑ends	Choose any of the supported inference engines – Ollama, llama.cpp, vLLM, MLX, Docker‑Model‑Runner, etc. – and Harbor will expose an OpenAI‑compatible endpoint for the front‑ends.
Rich UI catalogue	Over 20 front‑ends (Open WebUI, LibreChat, AnythingLLM, ComfyUI, Voicebox, …) are pre‑configured to talk to the chosen model.
Satellites & agents	Services like SearXNG (web search), Speaches (STT/TTS), Dify, Flowise, n8n, Boost‑agentic modules, etc., can be added with the same command and are automatically connected to the model.
Boost workflow engine	A lightweight “agentic” layer lets you chain modules (quickhop, deephop, autocheck, diffscope) into custom workflows that run before or after a model call.
Portable & exportable	harbor eject writes a plain docker‑compose.yml that reproduces the current setup, so you can move away from Harbor later.
Convenient helpers	QR‑code / URL shortcuts for phone access, built‑in Traefik reverse‑proxy, tunnelling (harbor tunnel) for internet exposure, command history, and per‑service config profiles.

---

Quick start (from the README)

# Install the CLI (script provided in the repo) – then:
harbor up               # spin up Open WebUI + a default LLM backend
harbor up searxng speaches   # add web‑search and voice capabilities

After the command finishes you can open the UI at http://localhost:3000 (or the port shown in the output) and start chatting, searching the web, or generating images.

---

Main feature groups

1. Local LLMs

• Run any supported backend (ollama, llamacpp, vllm, mlx, dmr, …) with a single command.
• Host‑native Metal/GPU inference on macOS (MLX/oMLX) – no containers needed for the compute.

2. Front‑ends (UIs)

• Open WebUI, LibreChat, AnythingLLM, ComfyUI, Voicebox, etc.
• All front‑ends automatically point to the running model endpoint.

3. Satellites & tooling

• Search – SearXNG, Perplexica, Morphic, Local Deep Research.
• Voice – Speaches (STT/TTS), Whisper‑based services.
• Image – ComfyUI + Flux integration.
• Workflow / automation – Dify, Flowise, LangFlow, n8n, LitLytics, etc.

4. Boost – agentic coding assistant

• Pre‑built modules (quickhop, deephop, autocheck, diffscope).
• Custom workflows defined via workflows.yaml or env vars.
• Can be invoked from the CLI: harbor launch --workflow deephop … codex.

5. Development helpers

• harbor launch – detects a running backend, injects its OpenAI‑compatible URL into tools like codex, claude, vscode, etc.
• Config profiles (harbor profile save …, harbor profile use …).
• History (harbor history), per‑service down/restart, and atomic .env repair.

---

Typical use‑cases

Use‑case	How Harbor helps
Experiment with a new LLM	Spin up ollama or llama.cpp with the desired model in seconds, then open Open WebUI to chat.
Add web‑search to a chatbot	harbor up searxng automatically connects SearXNG to the model; the UI can now perform RAG‑style searches.
Voice‑first assistant	harbor up speaches gives you speech‑to‑text and text‑to‑speech endpoints wired to the chat UI.
Run a local image‑generation pipeline	harbor up comfyui installs ComfyUI + Flux and makes it reachable from the same Open WebUI instance.
Build a multi‑step AI workflow	Define a Boost workflow (e.g., quickhop → autocheck → answer) and run it with harbor launch --workflow ….
Develop on a laptop without Docker for inference	Use the host‑native MLX/oMLX back‑ends on macOS; Harbor still provides the surrounding services via Docker.
Share a demo externally	harbor tunnel creates a temporary public URL (ngrok‑style) exposing any service, useful for quick demos.

---

Installation & getting started

1. Clone & install the CLI – the repo provides a script (install.sh) that adds the harbor binary to your $PATH.
2. Docker – Harbor relies on Docker Compose; ensure Docker Desktop (or the Docker engine) is running.
3. First launch – harbor up pulls the default stack (Open WebUI + a lightweight backend) and starts it.
4. Add services – e.g., harbor up ollama, harbor up searxng speaches, harbor up comfyui.
5. Optional UI – The companion “Harbor App” provides a graphical view of the stack and shortcuts for common commands.

---

Documentation & community

• Wiki – Detailed pages for installation, each service, Boost modules, and CLI reference.
• Discord – Active chat (discord.gg/8nDRphrhSF) for troubleshooting and feature requests.
• Release notes – Frequent updates (v0.5.x) adding new back‑ends, Boost modules, and stability fixes.
• Ko‑fi – Support the maintainers via the badge in the README.

---

TL;DR

Harbor is a one‑command orchestrator for a full local AI ecosystem. It hides Docker‑Compose plumbing, auto‑connects model servers to dozens of UIs and auxiliary services, and adds a lightweight agentic workflow layer (Boost). If you want to run LLMs, RAG, voice, or image generation on your own machine without fiddling with networking or env files, Harbor gives you a ready‑made, extensible stack.