A9T9/RPA

Ui.Vision – Browser & Desktop Automation (A9T9/RPA)

What it is – Ui.Vision is an open‑source browser extension (with optional desktop components) that lets you record, write, and run automation macros. It combines classic web‑automation (Selenium‑style commands), OCR/image‑recognition for visual tasks, and a Model Context Protocol (MCP) bridge that lets large‑language‑model assistants (e.g., Claude Code, Cursor) create or edit macros on your behalf.

Key capabilities

• Browser automation – fill forms, scrape data, download files, replay recorded actions.
• Visual automation – OCR to read on‑screen text and image‑matching to click elements when HTML selectors are missing.
• Desktop automation – via optional XModules you can control native apps or remote desktops.
• AI‑assisted scripting – an integrated AI assistant (or any MCP‑compatible client) can be asked to generate a macro; the assistant receives the uiv.* JavaScript API reference and returns a script that Ui.Vision executes.
• Scripting options – use the built‑in macro recorder, write plain‑text Selenium‑IDE commands, or program directly with the uiv.* JavaScript API (synchronous, no async/await).

How to get started

1. Install the extension from the Chrome Web Store, Edge Add‑ons, or Firefox Add‑ons (no build required for normal use).
2. Open Ui.Vision, record a simple task or load a demo macro.
3. For AI integration, install the MCP bridge with npx uivision-mcp-bridge --setup (requires Node ≥ 20 and npm). Enable the bridge in Ui.Vision → Settings → AI and connect your LLM client.
4. Ask the assistant a natural‑language command (e.g., “open https://example.com, read the heading, and save a macro”). The assistant will generate uiv.* calls, which you can review, edit, and run.

Developer notes

• The repo contains the source for the extension (React UI, command engine) and the MCP bridge.
• Build the extension locally with npm i -f && npm run build (Chrome/Edge) or npm run build-ff (Firefox). The output appears in dist/ or dist_ff/ and can be loaded as an unpacked extension.
• The mcp/ folder holds the bridge that translates MCP messages from LLM clients into macro actions.

Where to find more

• API & AI authoring guide: https://ui.vision/ai/ai-system-prompt
• Full plain‑text API for LLMs: https://ui.vision/llms-full.txt
• Selenium‑IDE command reference: https://ui.vision/rpa/docs/selenium-ide/
• Community forum: https://forum.ui.vision

Bottom line – Ui.Vision lets non‑programmers automate web and desktop workflows, and it uniquely exposes a protocol that lets modern LLM agents write and run those automations for you. It’s a genuine RPA tool with a clear AI‑assistant integration path.