droidrun/mobilerun

What it solves

Mobilerun solves the challenge of automating mobile devices using AI. It allows developers to control Android and iOS devices through natural language, enabling agents to perform tasks like navigating apps, inputting text, and extracting data without manual intervention.

How it works

The framework acts as a bridge between LLMs and physical or virtual mobile devices. By installing a companion 'Portal' app on the device, the framework gains the ability to inspect the screen, read the UI structure, and execute actions like taps and swipes. Users can trigger these actions via a command-line interface (CLI) or a Python API, with support for various LLM providers including OpenAI, Anthropic, and local models via Ollama.

Who it’s for

This tool is designed for developers and QA engineers who need to automate mobile app testing, perform repetitive mobile tasks, or build AI agents capable of interacting with native mobile environments.

Highlights

• Control Android and iOS devices using natural language commands.
• Supports both direct task execution and complex multi-step reasoning modes.
• Compatible with major LLM providers and local models.
• Offers flexible integration through CLI, Python API, or Docker containers.
• Includes vision-based understanding by combining screenshots with accessibility data.
• Provides built-in support for execution tracing and telemetry via tools like Langfuse.