ODS: a one-command local AI server stack that automates hardware detection and service orchestration

What it solves

ODS (Osmantic Deployment System) simplifies the process of setting up a private, local AI server. Instead of manually configuring multiple separate tools for inference, chat interfaces, and automation, ODS provides a single-command installation that wires together a complete stack of AI services on your own hardware.

How it works

ODS uses an automated installer that detects your GPU and hardware capabilities to automatically select the most appropriate LLM (Large Language Model) for your system. It deploys a suite of pre-configured services using Docker and native binaries (such as llama-server for macOS Metal acceleration). To minimize wait times, it uses a "bootstrap mode" that lets users chat with a small model immediately while the full-sized model downloads in the background.

Who it’s for

It is designed for individuals who want a sovereign, private AI environment at home, in a lab, or on a workstation without needing deep technical expertise in CUDA drivers or Docker configuration.

Highlights

• One-Command Setup: Automated installation for Linux, macOS (Apple Silicon), and Windows.
• Full-Service Stack: Includes Open WebUI for chat, llama-server for inference, n8n for workflows, and ComfyUI for image generation.
• Hardware Auto-Detection: Automatically maps VRAM/RAM to specific model tiers (e.g., Qwen, Phi, DeepSeek).
• Local-First Privacy: All data and prompts stay on the local machine by default, with optional cloud/hybrid modes via LiteLLM.
• Extensible Architecture: New services can be added as extensions via simple manifest and compose files.
• Integrated Tooling: Includes a control dashboard for GPU metrics and a CLI for stack management.