minitap-ai/mobile-use

AI agents can now use real Android and iOS apps, just like a human.

What it solves

Mobile-use is an AI agent designed to automate mobile devices (Android and iOS) using natural language. It eliminates the need for manual interaction by allowing users to perform complex tasks—such as sending messages or navigating apps—through simple text commands.

How it works

The system uses a multi-agent architecture to decompose tasks and interact with the device's user interface. It is UI-aware, meaning it can navigate app interfaces by leveraging accessibility tree data. It supports a wide range of LLM providers, including OpenAI, Google, xAI, OpenRouter, MiniMax, and Anthropic Claude, which can be configured to power the agent nodes.

Who it’s for

Developers and researchers interested in mobile automation, data scraping from mobile apps, and the development of agentic frameworks for mobile OS environments.

Highlights

  • Natural Language Control: Control your phone via native language commands.
  • UI-Aware Automation: Intelligently navigates app interfaces (though limited with games).
  • Data Scraping: Extract information from apps and structure it into formats like JSON via natural language descriptions.
  • Benchmark Performance: The first agentic framework to achieve 100% accuracy on the AndroidWorld benchmark.
  • Cross-Platform Support: Works with physical Android phones, Android simulators, and iOS simulators (macOS only).

Related

  • Project
    droidrun/mobile-harness

    A portable operating harness and Python API that enables AI agents to control Android and iOS devices locally or in the cloud.

  • Project
    droidrun/mobilerun

    An open-source framework that enables LLM agents to control Android and iOS devices using natural language, providing tools for UI inspection and mobile automation.

  • Project
    rounak/PhoneAgent

    An experimental mobile automation project that enables AI agents to control iOS and Android devices through an in-app agent or a JSON-RPC bridge.

  • Project
    Core-Mate/OpenGUI

    A mobile GUI agent framework for Android that enables AI agents to perceive and operate app interfaces on real devices for automated, long-running workflows.

  • Project
    xinzezhu/Open-AutoGLM-Android

    An AI-powered smartphone automation assistant that uses vision language models to execute natural language commands on Android via accessibility services.