droidrun/mobilerun-portal

What it solves

Mobilerun Portal provides a way to programmatically interact with and analyze Android user interfaces. It solves the difficulty of extracting real-time UI element data (like clickable or editable fields) and controlling a device remotely for purposes such as UI testing, automation development, and accessibility auditing.

How it works

It operates as an Android accessibility service that creates an interactive overlay on the screen to highlight UI elements. It exposes several control interfaces—including an HTTP socket server, WebSocket JSON-RPC, and an ADB-accessible ContentProvider—allowing external tools to query the UI tree, send keyboard inputs, and manage device state. It also supports a reverse WebSocket connection for cloud-based control and WebRTC for real-time screen streaming.

Who it’s for

This tool is designed for developers and QA engineers focusing on Android UI automation, accessibility testers, and those building remote-control or AI-driven mobile agent frameworks.

Highlights

• Multi-interface control: Supports ADB, HTTP, and WebSockets for local access, plus reverse WebSockets for cloud control.
• Real-time UI analysis: Extracts the accessibility tree as JSON, identifying clickable, scrollable, and focusable elements.
• Remote device management: Enables APK installation from URLs, clipboard manipulation, and screen-awake watchdogs.
• Visual feedback: Includes an interactive overlay to visually identify UI elements on the device screen.
• Low-level interaction: Provides direct keyboard input and key event simulation (e.g., Enter, Back, Home).