Osmantic/ODS

Turn your PC, Mac, or Linux box into an AI server. LLM inference, chat UI, voice, agents, workflows, RAG, and image generation.

What it solves

Running local AI often requires manually stitching together numerous complex services like Ollama, Open WebUI, and n8n, which can be difficult to configure and debug. ODS automates this entire process, turning a PC, Mac, or Linux machine into a private, sovereign AI server with a single command.

How it works

The system uses an intelligent installer that detects your hardware (NVIDIA, AMD, or Apple Silicon) and automatically selects the best AI models based on your available VRAM/RAM. It orchestrates a suite of services—including llama-server for inference, Open WebUI for chat, and Qdrant for vector storage—using Docker and native acceleration where available. It also features a "bootstrap mode" that provides an immediate, lightweight model while larger models download in the background.

Who it’s for

  • Privacy-conscious users who want to run AI locally without sending data to the cloud.
  • Homelab enthusiasts looking for a turnkey, pre-wired AI appliance.
  • Developers and researchers who need a managed local stack for testing agents, workflows, and RAG without manual configuration.

Highlights

  • Automated Hardware Detection: Automatically picks the optimal model tier based on your GPU or unified memory capacity.
  • All-in-One Stack: Includes text inference, image generation (ComfyUI), voice (Whisper/Kokoro), and workflow automation (n8n).
  • Agentic Capabilities: Features the Hermes Agent for autonomous tasks and OpenCode for browser-based coding assistance.
  • Extensible Architecture: Allows users to add new services as modular extensions via Docker Compose.
  • Management Tools: Includes a centralized dashboard for GPU metrics and a CLI for managing services and models.