cosmo-wander-ai/cosmo-edge

CosmoEdge 1.1 – Edge‑AI video analytics engine

What it is – A C++‑based runtime that lets you run video‑AI models (object detection, VLM‑style prompt‑driven vision, etc.) on a variety of edge processors – Sophon BM1688/BM1684X, Rockchip RK3576/RV1126B – as well as on ordinary x86/Linux, Windows, or macOS (Docker‑emulated) machines. The repository ships the core engine and a web console under Apache‑2.0; commercial‑grade model packages and hardware‑specific protection are offered separately.

Key capabilities

Capability	What you get
Native C++17 runtime	Multi‑channel video ingest, inference scheduling, on‑screen‑display (OSD), rule evaluation, alarm/event generation.
Visual orchestration UI	Browser‑based pipeline builder where you drag‑and‑drop models, set parameters, and see live feedback.
Multi‑backend inference	Sophon BMRT (.nn), Rockchip RKNN (.rknn), and ONNX Runtime (.onnn) back‑ends selected at build time.
Prompt‑driven vision (VLM/DINO)	Grounding‑DINO open‑vocabulary detection and large‑vision‑language model judgment that can trigger alarms.
Integration hooks	REST, WebSocket, MQTT, and generic HTTP webhooks for downstream systems.
Model onboarding & protection	Tools to convert, import, and validate models; “Model Guard” encrypts commercial presets for Sophon devices while keeping the same UI features.
Agent‑assisted development	Built‑in task entry points and checks that let a coding assistant (e.g., GitHub Copilot) help port models or extend the UI with verifiable outputs.

Supported platforms

• Sophon BM1688 (primary) – VLM at ~0.1 FPS per channel, up to 6 channels validated.
• Sophon CV186X – Same VLM performance as BM1688.
• Rockchip RK3576 – VLM supported; RKLLM v1.3.0 packaged.
• Rockchip RV1126B – Inference supported; VLM not yet validated.
• x86 Linux / Windows – ONNX Runtime; lower throughput, useful for development.
• macOS (Apple Silicon) preview – Docker linux/amd64 emulation for a single‑video demo.

Typical use cases

• Real‑time surveillance analytics (person detection, safety‑helmet checking, zone counting, etc.)
• Prompt‑driven visual inspection where a VLM decides whether a detection should raise an alarm
• Building custom video‑AI workflows without writing code, then exposing results via MQTT or webhooks for IoT or edge‑cloud integration

Quick start (x86 developer mode)

# clone the repo
git clone https://github.com/cosmo-wander-ai/cosmo-edge.git && cd cosmo-edge

# launch the Docker‑compose stack (Linux)
sudo docker compose -f docker-compose.x86.yml up -d --build
# Windows: docker compose -f docker-compose.x86.windows.yml up -d --build

# open the console
open http://localhost:8080   # or visit the URL in a browser

The same repo provides Docker‑based build scripts for Sophon and Rockchip devices; they pull the appropriate NPU SDK, compile the engine, and output a build_output/<chip>/ directory containing the runtime, model artifacts, a TARGET_CHIP marker, and SHA‑256 checksums.

Benchmarks – The v1.1 multi‑platform report (linked from the README) documents controlled 1080p24 tests:

• 5 FPS per business task across up to 8 concurrent channels
• 72‑hour continuous runs showing >5 FPS average on all supported chips
• VLM validation at a minimum of 0.1 FPS per channel (the gate used for PASS/FAIL)

Documentation & community

• Full docs site: https://www.cosmowander.ai/docs/
• Tutorials cover quick‑start, pipeline orchestration, VLM usage, and model porting.
• Support via GitHub Discussions, Issues, and a Gitee mirror for users in mainland China.
• Contributions accepted through pull requests; security issues reported privately via SECURITY.md.

License – Apache‑2.0 for the engine and console code. Commercial model packages and hardware‑specific protection layers are distributed under separate terms.

---

CosmoEdge is a genuine open‑source edge‑AI platform that bridges video model serving, visual workflow orchestration, and device‑specific acceleration, making it suitable for developers who need to deploy video analytics or VLM capabilities on low‑power NPU hardware.