A9T9/RPA

Ui.Vision Open-Source RPA Software with Computer Vision, OCR, Anthropic Computer Use/LLM. Selenium IDE import/export.

What it solves

Ui.Vision is a Robotic Process Automation (RPA) tool designed to automate repetitive tasks across web browsers and desktops. It replaces manual navigation and data entry by allowing users to create macros that can interact with web pages and the operating system.

How it works

The software operates as a browser extension for Chrome, Edge, and Firefox. It provides a Selenium IDE-compatible interface and a Web Macro Recorder for easy automation creation. For advanced users, it offers a modern JavaScript API (uiv.*) that enables DOM and visual finders, OCR, and desktop interaction. It also integrates with AI assistants via a system prompt and an MCP bridge, allowing AI clients like Claude Code to create and run macros directly.

Who it’s for

It is intended for users who want to automate browser-based and desktop workflows, ranging from non-experts using the macro recorder to developers who prefer writing JavaScript-based automation scripts.

Highlights

  • Cross-platform support: Works on macOS, Linux, and Windows via major browsers.
  • AI Integration: Includes a dedicated AI system prompt and an MCP bridge for AI-driven macro generation.
  • Visual Automation: Supports visual finders and OCR for interacting with elements that aren't easily accessible via the DOM.
  • Open-source and free: Available for personal and commercial use.

Related

  • Project
    aisingapore/TagUI

    An open-source robotic process automation (RPA) tool that lets users automate web, desktop, and command-line tasks using a simple, natural-language syntax.

  • Project
    oculix-org/SikuliX1

    A computer vision-based automation tool that identifies and interacts with screen elements using image recognition to simulate mouse and keyboard input.

  • Project
    pmh1314520/WebRPA

    WebRPA is an open‑source, Windows‑only visual RPA platform that lets users build web, desktop and Android automation flows by dragging 571 pre‑made modules. It includes a bundled Python 3.13 + Node 20 runtime, Playwright for browser control, extensive AI features (chat, OCR, vision, speech, data extraction) via OpenAI‑compatible APIs, and an AI “assistant” that can auto‑create and self‑heal workflows. Optional heavy‑weight capabilities (OCR, media processing, mobile automation, QQ bots, etc.) are delivered as downloadable feature packs. The UI is built with React 19, TypeScript, Tailwind and React Flow; the backend uses FastAPI. Licensed under AGPL‑3.0 for free personal use, with a commercial license available for closed‑source deployments.

  • Project
    web-infra-dev/midscene

    A vision-driven GUI agent for E2E testing and UI automation that uses multimodal AI to interact with interfaces via screenshots and natural language instead of fragile DOM selectors.

  • Project
    tylergraydev/claude-code-tool-manager

    A desktop application that provides a visual interface to manage MCP servers, commands, and agents across multiple AI coding assistants like Claude Code and Cursor.