leeguooooo/iphone-use

What it solves

iphone-use allows AI agents to interact with a real iPhone as if they were a human user. It enables agents to navigate apps, tap buttons, type text, and swipe screens, even for apps that lack an official API or block standard screen recording.

How it works

A daemon runs on your Mac and connects to your iPhone via USB or Wi-Fi. It installs a specialized XCTest runner on the phone that reads the screen as an accessibility tree (text-based) rather than just pixels. This allows the agent to "see" and interact with UI elements directly. The project acts as an MCP (Model Context Protocol) server, allowing AI clients like Claude Code or Codex to send commands to the phone and receive feedback on whether actions were successful.

Who it’s for

This tool is for developers, testers, and power users who want to automate tasks on a physical iPhone using AI agents. It is particularly useful for those who need to perform repetitive workflows, run automated tests, or control their phone remotely from a computer.

Highlights

• Works with any app, including those without APIs or with screen-capture protections.
• Supports both USB and Wi-Fi connections for flexibility.
• Includes a "Flows" feature to record and replay multi-step tasks without using AI tokens.
• Provides a live, remote-control interface for humans to use the phone from a browser or another iPhone.
• Built for speed, with high-performance screen reading and action execution.
• Prioritizes privacy by keeping secrets (like passwords) on the device and allowing users to block specific apps from agent access.