dsh‑vision‑router

What it is – A plug‑in for the DeepSeek Harness (DSH) chat UI that lets a text‑only LLM (e.g., DeepSeek) “see” images on demand. It routes image turns through a configurable vision model (local or cloud) and exposes a set of vision‑related tools (vision_describe, vision_ground, vision_crop, …) that the LLM can call just like any other tool.

Why it matters – Most existing DSH vision plug‑ins only convert an image to a short text description before sending it to the LLM, losing pixel‑level detail. dsh‑vision‑router keeps the original raster data on the vision‑model side, lets the LLM request fine‑grained operations (crop, OCR, pixel‑diff, etc.), and falls back to a free, key‑less OV‑Hcloud endpoint when no user‑provided model is configured. This gives developers a way to build multi‑step visual workflows without paying for a vision API or writing Python code.

How it works

• The plug‑in is installed with a single npx @deepseek‑ai/dsh plugin … add dsh‑vision‑router command; it automatically patches the DSH bundle so the new row, admission wrapper and attachment limits are added.
• When the user toggles the 👁 Vision button in the composer, DSH switches the current session to a special “vision wrapper” route that intercepts image turns.
• The wrapper decides, per‑turn, whether to handle the image locally (crop, OCR via Tesseract, palette, SVG trace, etc.) or forward the pixel data to a vision provider.
• Providers can be:
  • a user‑configured cloud endpoint (OpenAI‑compatible, Ollama, LM Studio, etc.),
  • the built‑in free fallback that sends the request anonymously to OVHcloud AI Endpoints, or
  • a local tool chain that never leaves the machine.
• Results are returned to the LLM as JSON‑structured tool output, cached by image content, and can be used in subsequent tool calls (e.g., ground → crop → describe → pixel‑diff).
• All heavy‑lifting is done with Node‑native libraries (sharp, potrace, tesseract) and the system Chrome; no Python runtime is required.

Key features

• One‑command install – No manual file edits; the plug‑in patches DSH automatically.
• Free out‑of‑box vision – Anonymous OVHcloud endpoint (2 req/min per IP per model) works without an API key.
• No Python – Entire pipeline runs on Node.js (≥ 22).
• Multi‑step visual toolset – crop, OCR, palette extraction, SVG trace, HTML screenshot, pixel‑diff, grounding, etc.
• Local‑only mode – A runtime switch that blocks any remote calls, useful for confidential data.
• Automatic fallback chain – If the primary provider fails, the plug‑in falls back to the next configured backend.
• Caching – Vision results are cached per‑image, avoiding duplicate calls.
• Transparent UI – Images stay displayed as images; the vision routing happens behind the scenes.

Typical workflow

1. Install the plug‑in via the single npx command.
2. Open DSH (Web or Desktop) and enable the 👁 Vision toggle.
3. Paste or upload an image.
4. The LLM can now call vision tools (e.g., vision_ground) to locate objects, vision_crop to extract a region, vision_ocr for text, etc.
5. The agent iterates with tool calls until the visual task is complete, then replies with a structured answer.

When to use it – When you need pixel‑accurate visual reasoning (UI automation, image editing, OCR, object grounding) inside a DeepSeek‑based chat without writing custom Python wrappers or paying for a vision API. It is also handy for prototyping multi‑turn visual workflows that stay within the DSH ecosystem.

---

Links & resources

• GitHub repo – source, issues, releases (v2.3.0 latest).
• Installation docs – one‑liner npx … add dsh-vision-router.
• Data‑flow & privacy note – explains remote data transmission and the local‑only switch.
• Demo GIF – shows an image turn, vision grounding, cropping and the final answer.
• Community: QQ group 1105463028, various DSH plugin marketplaces.

---

This summary is based solely on the repository’s README.