ShawnPana/phone-harness

What it solves

Phone Harness allows AI agents (such as Claude Code or Codex) to control a physical or cloud-based smartphone. It removes the need for jailbreaking or installing software on the device, enabling agents to interact with mobile apps by seeing the screen and performing actions like tapping and typing.

How it works

Depending on the platform, the tool uses different transport methods:

• iPhone: Uses Mac's iPhone Mirroring to render the phone as a window, Apple's Vision framework for OCR to find text coordinates, and HID-level events for input.
• Android: Uses Android Debug Bridge (adb) to capture the screen and access the accessibility tree for text, using the input command for actions.
• Cloud Phones: Provides hosted Androids (via adb) or iPhones (via HTTPS) that can be controlled remotely from any operating system.

Who it’s for

Developers and AI researchers who want to give their coding agents the ability to automate tasks on real mobile devices, such as testing apps or performing real-world actions like booking a ride.

Highlights

• Multi-platform support: Works with iPhone, Android, and cloud-hosted devices.
• No device modification: Requires no jailbreak, Xcode, or on-device app installation.
• Agent-ready: Includes a setup prompt and skill definitions to easily integrate with AI agents.
• OCR-driven interaction: Uses screen capture and text recognition to identify where to tap and type.