Yzw202011/OmniSpace

OmniSpace AI – A Local‑First, Full‑Stack Creative AI Studio

What it is – OmniSpace is a desktop‑focused platform that stitches together a variety of generative‑AI models (text, image, video, voice, and knowledge‑graph) into a single, engineered workflow. All inference runs on the user’s own GPU by default; optional cloud APIs can be added with a key.

Core capabilities

Area	What you can do	Key tech
AI Comic	Generate multi‑panel comics, keep character style consistent with LoRA, auto‑split scripts into panels, choose five speech‑bubble shapes, edit assets (characters, scenes, props) with SAM‑based cut‑out, export PNG or PDF	FLUX.2‑Klein, SAM, custom UI
AI Manga / Video	End‑to‑end storyboard → asset → key‑frame → video pipeline. Uses MiniMax‑H3 chain and ComfyUI for local video generation, with queue, progress UI and style packs	MiniMax‑H3, ComfyUI, vLLM/llama.cpp back‑ends
AI Dialog	Multi‑engine local chat (vLLM, llama.cpp, Transformers), streaming via WebSocket, context + RAG from a vector store, file/image upload, auto‑downgrade when GPU memory is low	vLLM 0.27.1, ChromaDB, FTS5
Novel Writer	Outline → chapter generation with length & style controls (standard / light‑AI / heavy‑AI), character & world‑building assistance, export to TXT/MD	Transformers, custom prompting
Learning / RAG	Ingest documents & images into a vector+full‑text index, visual knowledge‑graph, usage analytics, quota alerts	ChromaDB, RRF fusion
Voice	Local TTS (Windows SAPI5) and Whisper transcription, custom voice upload; voice cloning planned	Whisper, local TTS stack
Plugin system	"CuteMamen" packages (tar.gz) can add new inference engines or data‑only extensions. Packages are sandboxed (sub‑process, AST checks, memory limits) and carry trust badges.	Rust sandbox, AST security
Ops & safety	VRAM scheduler for 16 GB cards, module isolation with self‑healing ladders, encrypted backups (AES‑GCM), audit logs, power‑loss recovery, tiered delete confirmations, hardware health dashboard	Custom engine, SQLite + WAL, DPAPI

Architecture snapshot

• Frontend – React 19 + TypeScript, Vite, Tailwind, Zustand, hash‑router, six theme families.
• Backend – FastAPI 0.141.1, Pydantic v2, uvicorn, runs on an embedded Python 3.12 runtime.
• Inference layer – PyTorch 2.13 (CUDA 13.0) with Diffusers & Transformers; hot‑switchable back‑ends (vLLM, llama.cpp GGUF).
• Data – SQLite + ChromaDB for vectors, FTS5 for full‑text, AES‑256‑GCM field encryption.
• Deployment – A launcher daemon starts services, opens a browser at http://127.0.0.1:8765. Optional Tauri WebView shell.
• Testing – 1 132 pytest cases (GPU‑aware) + 101 vitest UI tests, extensive CI checks.

Getting started (Windows)

cd E:\OmniSpace
# simple double‑click start script or:
runtime\py312\python.exe launcher\launcher.py   # launches FastAPI + UI

Then open the auto‑opened browser. Development mode runs the API with uvicorn src.main:app --reload and the UI with pnpm dev.

Who might use it?

• Content creators who want a single app for script‑to‑comic/video generation without sending data to the cloud.
• AI researchers / engineers looking for a ready‑made multimodal pipeline to plug new models into.
• Teams that need strong local data‑privacy, audit trails, and automated recovery from GPU OOM or crashes.

Limitations & roadmap

• Currently Windows‑only; Linux/macOS support is planned.
• Voice‑cloning is still in research.
• Multi‑GPU scaling and online search integration are in progress.

Bottom line – OmniSpace is a genuine, engineering‑heavy open‑source‑ish project that bundles multimodal generation, RAG, and robust ops tooling into a locally‑run creative studio.