Osmantic/ODS

ODS V3 Pre-Release: Public testing and refinement ahead of the official V3 launch. Turn your PC, Mac, or Linux box into a private AI server.

What it solves

ODS (Osmantic Deployment System) eliminates the complexity of setting up a private, local AI server. Instead of manually installing and configuring multiple disparate tools for inference, chat interfaces, and automation, ODS provides a single-command installation that wires these services together into a cohesive, private AI stack on your own hardware.

How it works

ODS uses a series of installers (bash for Linux/macOS and PowerShell for Windows/WSL2) to deploy a suite of containerized services. It automatically detects the user's GPU (NVIDIA, AMD, or Apple Silicon) to select the appropriate inference engine (such as llama-server) and model. The system integrates several open-source components—including Open WebUI for chat, Qdrant for vector storage, and n8n for workflows—and manages them through a centralized control dashboard for monitoring GPU status and service health.

Who it’s for

It is designed for individuals, homelab enthusiasts, and professionals who want to run private AI locally on their workstations or servers without needing deep expertise in CUDA drivers, Docker configurations, or manual API wiring.

Highlights

  • One-Command Setup: Automatically detects hardware and configures the entire stack.
  • Private by Default: No cloud requirements or telemetry; data and chat history stay on the local machine.
  • Comprehensive Toolset: Includes local LLM inference, image generation (ComfyUI), speech-to-text (Whisper), and text-to-speech (Kokoro).
  • Agentic Workflows: Integrates n8n for automation and includes specialized agents like Portal and Hermes.
  • RAG and Search: Built-in support for retrieval-augmented generation via Qdrant and private web search via SearXNG.
  • Cross-Platform: Supports Linux, macOS (Apple Silicon), and Windows (via WSL2/Docker Desktop).

Related

  • Project
    OpenVoiceOS/ovos-installer

    A one‑command installer that sets up the open‑source Open Voice OS voice assistant (with optional local speech, Home Assistant integration, LLM fallback, and multi‑device HiveMind support) on Linux or macOS devices.

  • Project
    osaurus-ai/osaurus

    Osaurus is a native macOS app that lets you run private AI agents locally. Agents have their own memory, can execute code in an isolated Linux VM or macOS sandbox, call tools (file system, browser, Apple apps, image generation), and communicate securely via encrypted channels. The platform works with local models (MLX, Apple Foundation Models, Liquid AI) or cloud APIs (OpenAI, Anthropic, etc.), and includes privacy‑filtering, cryptographic identities, plugin support, voice input, and system‑wide shortcuts. Install via Homebrew or a .dmg; requires macOS 15.5+ on Apple Silicon.

  • Project
    beclab/Olares

    Olares is an open‑source personal‑cloud operating system that runs on your own Linux machine (or cluster). Built on Kubernetes, it offers a web UI and a suite of system apps (Files, Vault, Market, Dashboard) plus AI‑specific agents (Lares, OpenClaw). With one‑click model installation, GPU pooling, private networking, and CLI “Agent Skills,” it lets individuals or small teams host LLMs, generative‑AI tools, home‑automation, and collaborative workspaces while keeping data and compute local.

  • Project
    jaylfc/taOS

    taOS is a self-hosted AI agent platform that turns spare hardware (laptops, Pis, gaming PCs, phones) into a private distributed compute cluster. It offers a full web desktop environment, framework-agnostic agents, a self-hosted memory system (taOSmd), and large app/model catalogs—all offline-first with cloud as an opt-in. It's beta software with some rough edges and untested install manifests.

  • Project
    onyx-dot-app/onyx-foss

    Onyx is a self-hosted knowledge and context layer that connects to over 50 applications to provide AI agents with secure, relevant access to your organization's internal data.