Light-Heart-Labs/ODS

Turn your PC, Mac, or Linux box into an AI server. LLM inference, chat UI, voice, agents, workflows, RAG, and image generation.

What it solves

ODS (Osmantic Deployment System) simplifies the process of setting up a private, local AI server. Instead of manually configuring multiple separate tools for inference, chat interfaces, and automation, ODS provides a single-command installation that wires together a complete suite of AI services on your own hardware.

How it works

ODS uses an automated installer that detects your GPU (NVIDIA, AMD, Intel Arc) or Apple Silicon chip to automatically select the best-fitting LLM model based on your available VRAM/RAM. It deploys a pre-configured stack of services—including llama-server for inference, Open WebUI for the chat interface, and Qdrant for RAG—using Docker and native binaries. It also features a "bootstrap mode" that allows users to start chatting with a small model immediately while the full-sized model downloads in the background.

Who it’s for

It is designed for individuals who want a private AI homelab, workstations, or lab environments without needing deep technical expertise in CUDA drivers or Docker configurations.

Highlights

  • One-command setup: Rapidly deploys a full AI stack on Linux, macOS, and Windows.
  • Curation of AI tools: Integrates Open WebUI, n8n for workflows, ComfyUI for images, and Whisper/Kokoro for voice.
  • Hardware auto-detection: Automatically maps hardware tiers to the most appropriate GGUF models.
  • Extensible architecture: Allows users to add new services as extensions via simple manifests.
  • Privacy-first: Runs entirely locally with no cloud subscriptions required, though optional hybrid/cloud modes are available.