MiaAI-Lab/sparkDash

sparkDash – Real‑time monitoring dashboard for NVIDIA DGX Spark and other GPU hosts

What it is

• A web‑based dashboard (React front‑end, Express back‑end, Docker‑first) that lets you watch the health and performance of one or many NVIDIA DGX Spark (GB10) machines and any other Linux/Windows host with an NVIDIA GPU.
• It gathers metrics over SSH (or native OpenSSH on Windows) and streams them to the browser via WebSockets, so you see live graphs of GPU/CPU load, memory usage, network, storage, energy consumption and even LLM‑specific counters.

Key capabilities

Feature	Why it matters for AI workloads
Multi‑unit monitoring	Track a fleet of DGX Sparks or mixed GPU servers from a single UI; add, edit, reorder or remove units without restarting the dashboard.
LLM inference metrics	Auto‑detects common inference servers (llama.cpp, vLLM, sglang, LM Studio, Ollama, TensorFold, etc.) and shows live decode/prefill token‑per‑second rates, cache usage, queue length, TTFT/E2E latency, and token totals per model.
Benchmarks	Built‑in decode, prefill, quality (GSM8K, MMLU) and tool‑eval suites let you measure throughput and accuracy of the hosted models directly from the UI.
GPU‑process view	Lists top GPU‑using processes with VRAM consumption, helping you spot runaway jobs.
Unified memory & RDMA monitoring	For DGX Spark it shows unified memory split, RoCE/RDMA link health, traffic and drops – crucial for high‑speed multi‑node training.
ComfyUI & Hermes Agent integration	Optional probes let you watch Stable‑Diffusion job queues (ComfyUI) and keep a Hermes Agent up‑to‑date across the fleet.
Cross‑platform hosts	Supports Linux, macOS and Windows (including AMD‑iGPU Windows hosts) via SSH or PowerShell, with separate RAM/VRAM panels for non‑Spark machines.
Mobile‑friendly PWA	Installable as a Progressive Web App, so you can monitor your cluster from a phone over HTTPS (e.g., via Tailscale Serve).
Prometheus export	Opt‑in /metrics endpoint for integration with existing observability stacks.
Energy tracking	Shows per‑unit and fleet‑wide energy usage, with daily/monthly history and optional cost estimation.
Security	SSH passwords are stored encrypted (AES‑256‑GCM); secrets never appear in configuration files or API responses.

How it works

• Collectors run on each monitored host (Linux/macOS use nvidia‑smi and /proc data; Windows uses OpenSSH + PowerShell counters). They push JSON snapshots to the central Express server.
• The server aggregates the data, keeps a short‑term history for sparklines, and serves it via a WebSocket stream to the React UI.
• Configuration (sparks.json) is edited through the UI; changes are applied instantly without restarting the container.

Who would use it

• AI research labs or enterprises that run multiple DGX Spark nodes and need a single pane‑of‑glass for hardware health, energy consumption, and LLM inference performance.
• Teams deploying on‑premise GPU servers (e.g., RTX workstations) who also want to monitor LLM services, Stable‑Diffusion queues, or custom inference back‑ends.
• Ops engineers looking for a lightweight, Docker‑based alternative to full‑blown monitoring stacks while still being able to export Prometheus metrics.

Getting started

1. Clone the repo and run the provided Docker Compose files (dev or prod).
2. Use the Add Spark wizard in the UI to point at a host (SSH key or password).
3. Optionally enable LLM server presets (LM Studio, Ollama, etc.) and other services (ComfyUI, Hermes).
4. Watch the live dashboards, run benchmarks, and export metrics as needed.

License: Apache 2.0 – free for commercial and research use.

---

All details are taken directly from the repository’s README; no additional features are inferred.