cosmo-wander-ai/cosmo-edge

Production-grade C++ edge AI engine for video analytics and on-device VLM across Sophon, Rockchip RKNN, and x86, with visual orchestration, real-time OSD, events, and reproducible benchmarks.

What it solves

CosmoEdge is a C++ edge AI engine designed to turn video AI models into deployable applications. It simplifies the process of deploying video analytics, Vision Language Models (VLMs), and event workflows across different hardware platforms (Sophon, Rockchip, and x86) by providing a consistent orchestration experience despite the different runtimes and model artifacts required by each.

How it works

The system consists of a C++ engine core that handles media, inference scheduling, and event management, and a web console for visual orchestration. Users can compose pipelines in the browser, binding models, rules, and output actions. The engine supports multiple backends including Sophon BMRT/VPU, RKNN, and ONNX Runtime, allowing the same workflow to be adapted to different edge devices.

Who it’s for

It is intended for developers and engineers building real-time video analytics and prompt-driven vision applications for edge hardware, specifically those using Sophon or Rockchip NPUs.

Highlights

  • Multi-platform support: Native support for Sophon (BM1688, CV186X) and Rockchip (RK3576, RV1126B) NPUs, as well as x86 Linux/Windows.
  • Visual Orchestration: A browser-based interface for composing AI pipelines without needing to write code for every workflow.
  • VLM and Open-Vocabulary Detection: Support for prompt-based visual judgment and GroundingDINO for open-vocabulary detection.
  • Edge Integration: Delivers structured events via REST, WebSocket, MQTT, or HTTP webhooks.
  • C++17 Engine: High-performance native runtime for multi-channel media and inference scheduling.

Related

  • Project
    omnimind-ai/OmniInfer

    OmniInfer is a cross‑platform inference engine that lets you run LLMs and vision‑language models locally via a simple CLI and OpenAI‑compatible HTTP server. It supports many backends (llama.cpp, MLX, MNN, etc.), works on desktop, Android and iOS, and provides hardware‑aware optimisation and a terminal UI for chatting.

  • Project
    Tencent/AI-Infra-Guard

    AI‑Infra‑Guard (A.I.G) is an open‑source, Docker‑deployable platform for red‑team security testing of AI services. It scans live LLM endpoints, AI‑agent workflows, and model‑control‑plane code for CVEs, jailbreak susceptibility, and agent loss‑of‑control risks, exposing results via a web UI and REST API.

  • Project
    VIAME/VIAME

    VIAME is an open‑source, pipeline‑based computer‑vision toolkit that bundles detection, tracking, annotation, measurement, and model‑training tools. It offers both desktop and web GUIs, a Python package, Docker images, and optional GPU acceleration, making it suitable for DIY AI video analytics across many domains.

  • Project
    xtreme1-io/xtreme1

    Xtreme1 is an open‑source, Docker‑based platform for annotating, curating, and managing multimodal training data (2‑D images, 3‑D LiDAR, sensor‑fusion). It offers UI‑driven labeling tools, ontology management, built‑in pre‑labeling models (YOLOR, OpenPCDet), quality monitoring, and a beta RLHF interface for LLMs. Installable via a single `docker compose up` command, it runs on any OS with Docker and can optionally use GPU‑accelerated model containers. Licensed under Apache‑2.0 and hosted by the LF AI & Data Foundation.

  • Dispatch
    Qwen 3.5‑397B‑A17B release: hybrid linear‑attention MoE model with 1 M token context and state‑of‑the‑art multimodal performance

    Qwen 3.5‑397B‑A17B is a 397 billion‑parameter multimodal model that activates only 17 billion parameters per token, delivering state‑of‑the‑art performance on language, coding, reasoning and vision tasks while being up to 19× faster than its predecessor.