Offline inference engine for art, real-time voice conversations, LLM powered chatbots and automated workflows

AI Runner (airunner)

What it is

AI Runner is a self‑hosted desktop application that combines two main experiences:

AI companion – a chat‑bot you can name, give a voice to, and let remember you over time. It can talk, listen (speech‑to‑text), speak (text‑to‑speech), and stays aware of the current time, date and weather.
Art canvas – a layered drawing surface where you can sketch, paint, then turn those sketches into images with local diffusion models, edit them (in‑painting, background removal, style filters) and composite layers.

All processing (LLM chat, image generation, TTS/STT) runs on the user’s own hardware – no API keys, no cloud subscription, and nothing leaves the machine unless the optional cloud features are turned on.

Key features (as described in the README)

Feature	What you get
🤖 AI Companion	Named, voiced personality with persistent memory and mood that shifts naturally.
🎨 Layered Canvas	Sketch → generate → filter → composite on multiple layers.
🖼️ Image Generation	Z‑Image Turbo, Stable Diffusion 1.5, image‑to‑image, in‑painting, background removal, post‑process filters.
🗣️ Voice Conversation	Full‑duplex TTS (text‑to‑speech) and STT (speech‑to‑text) with selectable voices.
🧠 Memory & RAG	Long‑term conversation memory; retrieval‑augmented generation for recalling past talks.
🌤️ Environmental awareness	Knows local time, date and weather, which can influence responses.
🔒 Privacy‑first	All data stays locally; optional cloud features are opt‑in only.
🛡️ Safety filters	Built‑in NSFW filter and illegal‑content classifier.
📦 Model management	One‑click download from HuggingFace or CivitAI; supports multiple local LLM and image models.
🌍 Multilingual	English (full stack), Japanese (TTS/LLM), limited STT for other languages.

How you can use it

Mode	Typical use‑case
Desktop GUI (installed via the Itch.io download, Docker, or `pip install airunner`)	Interactive chat with a voice‑enabled companion while you draw and generate art on the same screen.
Headless server (`airunner‑headless`)	Run a REST API that exposes LLM, image, TTS and STT endpoints – useful for remote servers, VS Code extensions, or automation pipelines.
Docker container	Quick sandboxed deployment; can run either the full GUI (`docker compose run …`) or the headless API (`--headless`).
Python library	Import `airunner` packages in your own scripts to call the same inference back‑ends.

Installation options (summarised)

Method	What you get	Notes
Desktop download	Pre‑built binary (no Python needed).
Docker	Container image with optional profiles (`core,llm-native,stt-native` etc.).
PyPI (`pip install airunner`)	Python package that pulls in the GUI and the `airunner‑services` runtime.
APT (Ubuntu 22.04+)	Installs only the daemon services; no Qt GUI.
From source	Clone repo, run `./scripts/install.sh` (dev mode) or `./deployment/install_distributed.sh` for split daemon / GUI installs.

All paths eventually install the same set of side‑car binaries (llama.cpp for LLM, whisper.cpp for STT) which are built or downloaded automatically.

Architecture at a glance

flowchart LR
  User --> Native[Native launcher & side‑cars]
  Native --> GUI[Desktop client (src)]
  Native --> Daemon[Headless daemon (services)]
  GUI --> Daemon
  Native --> Sidecars[llama.cpp & whisper.cpp]
  Daemon --> Sidecars
  GUI --> Scripts[dev tools]
  Daemon --> Scripts
  GUI --> Data[(AIRUNNER_BASE_PATH)]
  Daemon --> Data

src – Qt‑based GUI, widgets, entry point.
services – FastAPI server, model download manager, persistence, safety filters.
native – Small helper that bundles the compiled side‑cars and launches the appropriate entry point.
scripts – Helper commands for building UI, running tests, generating certificates, etc.

Model support (default models)

Type	Default model (size)
LLM	Qwen3.5‑9B GGUF (`Q8_0`, ~12 GB VRAM)
Image	Stable Diffusion 1.5 (~~2 GB) and Z‑Image Turbo (~~12 GB)
TTS	OpenVoice (≈654 MB)
STT	`Systran/faster‑distil‑whisper‑large‑v3`

You can replace any of these by placing a compatible file in ~/.local/share/airunner/... or by passing a path to the CLI/GUI download manager.

Who maintains it?

The project is released under GPL‑3.0 and is primarily developed by a single engineer at Capsize LLC (https://capsize.online). Community support happens on Discord (link in the badge) and via the GitHub issue tracker.

Quick start (one‑liner for most users)

# Linux/macOS – install the Python package and run the GUI
pip install "airunner" "airunner-services[headless]"
airunner   # opens the desktop app

For a headless API server:

airunner-headless   # listens on http://127.0.0.1:8080

Bottom line

AI Runner is a genuine, self‑hosted AI‑assistant + generative‑art platform. It bundles local LLM inference (via llama.cpp), diffusion image generation, and speech pipelines, all wrapped in a privacy‑first desktop UI that can also be exposed as a REST service.

Capsize-Games/airunner