localgpt-app/localgpt

Local AI assistant, dreaming explorable worlds.

LocalGPT – AI‑first 3‑D world builder & local‑only assistant

What it is – A Rust‑based, single‑binary application that lets you:

  • Generate explorable 3‑D scenes from natural‑language prompts using the Bevy game engine (localgpt‑gen).
  • Run a local AI assistant with persistent, searchable memory, autonomous background tasks, and a tiny HTTP API (localgpt).

Both components run entirely on your machine – no Docker, Node, or Python required – and can talk to any LLM you point them at (Ollama, LM Studio, OpenAI, Anthropic, etc.).

Quick start (users)

# install the world‑builder binary
cargo install localgpt-gen
# install the assistant binary
cargo install localgpt

Run a prompt‑driven scene:

localgpt-gen "Create a desert scene with pyramids and a UFO hovering above"

Start the chat assistant:

localgpt chat          # interactive REPL
localgpt ask "What is the meaning of life?"
localgpt daemon start  # launches HTTP API + web UI

Core features

Component Highlights
Gen mode Parametric primitives, PBR materials, lights, behaviours (orbit, spin, path‑follow), ambient audio, export to glTF/GLB or HTML, headless batch mode, MCP server for IDE integration
AI Assistant Persistent markdown‑based knowledge store, full‑text + semantic search (SQLite FTS5 + sqlite‑vec), autonomous heartbeat task queue, multi‑interface (CLI, web UI, desktop GUI, mobile), sandboxed execution (Landlock/seccomp, Seatbelt), prompt‑injection defenses
LLM support Works with any local or cloud provider – LM Studio, Ollama, Anthropic, OpenAI, xAI, GLM, Vertex AI, plus generic CLI providers
Security Kernel‑enforced sandbox, HMAC‑signed policy files, audit‑chain log
Extensibility Skill system – you can save a whole world as a reusable “skill”, and the memory system learns your style (palettes, lighting, templates) across sessions

Typical use cases

  • Rapid prototyping for games or VR – describe a scene in plain English and get a ready‑to‑run Bevy project.
  • Creative brainstorming – iterate over many variations in headless mode for batch generation or CI pipelines.
  • Personal knowledge assistant – keep notes, code snippets, or research in a local, searchable memory that can be queried via CLI or HTTP.
  • Automation – let the daemon’s heartbeat run background tasks (e.g., fetch web data, update a world) without user interaction.

How it works under the hood

  • Rust + Tokio – async runtime for both the daemon and the generator.
  • Bevy – ECS‑based game engine that constructs the 3‑D scene.
  • SQLite – stores markdown memory; FTS5 for keyword search, sqlite‑vec for embedding‑based semantic search (embeddings produced by fastembed).
  • Axum – lightweight HTTP server exposing the web UI and JSON endpoints.
  • eframe – optional native GUI front‑end.

Documentation & community

License

Apache‑2.0 – free for personal and commercial use.

Related

  • Project
    fy-agent/fyagent

    FyAgent is a cross‑platform desktop app that consolidates the configuration of AI‑assistant tools (model/provider selection, skill/plugin installation, prompt and memory editing, and service credentials) into a single local UI, helping users manage multiple AI agents consistently.

  • Project
    Deep-ai-inc/ch.at

    ch.at is a single‑binary Go server that lets you chat with an LLM (default GPT‑4o) via HTTP, SSH, DNS, or API, with no JavaScript, no accounts, and all state kept in RAM. It’s privacy‑focused, easy to self‑host, and includes a public board for posting messages over the same lightweight protocols.

  • Project
    agentrq/agentrq

    AgentRQ is an open‑source platform that lets humans and LLM agents (Claude, Codex, Antigravity, etc.) share a real‑time task workspace. It provides a web UI and native desktop app, a Go‑based backend exposing a Model Context Protocol (MCP) that agents call to create/update tasks, fire events, read/write notes, and more. Features include a Kanban board, scheduled/recurring tasks, drag‑and‑drop event‑driven workflows, on‑device auto‑title generation, speech‑to‑text, and extensive keyboard shortcuts. Extensions can add custom pages or actions. The repo includes Docker/self‑host instructions, a one‑liner desktop installer, and guides for integrating Claude via `.mcp.json` and for bridging other agents through an ACP gateway.

  • Project
    vndee/llm-sandbox

    LLM Sandbox is a Python package that runs code generated by large language models inside isolated containers (Docker, Kubernetes, or Podman). It supports multiple languages, enforces security policies, can capture output plots, offers interactive notebook‑style sessions, and includes a container‑pooling system for fast, concurrent execution.