ghost-in-the-droid/android-agent

What it solves

Ghost in the Droid provides a unified framework for AI agents to interact with real mobile devices. It solves the problem of "brain-only" AI agents that can think but cannot physically interact with phone interfaces, allowing them to read screens, tap buttons, swipe, and type across both Android and iOS platforms.

How it works

The project acts as a "body" for any LLM. It uses ADB (Android Debug Bridge) for Android devices and Appium/WebDriverAgent for iOS devices to execute commands. It exposes a set of 62 tools via the Model Context Protocol (MCP), enabling compatible clients (like Claude Code or Cursor) to control the phone.

To optimize costs and speed, it uses "skills"—deterministic YAML and Python recipes for known workflows—so the agent doesn't have to use expensive LLM tokens for repetitive tasks. It also supports on-device inference using engines like llama.cpp and MediaPipe, allowing the agent to run entirely on the phone in airplane mode.

Who it’s for

• AI Developers: Those building agents that need to operate mobile apps.
• Automation Engineers: Users wanting to scale mobile automation across phone farms.
• LLM Researchers: People testing on-device LLM capabilities for mobile OS control.

Highlights

• Cross-Platform: Supports both Android and iOS (real devices and simulators).
• MCP Integration: Provides 62 tools for vision, touch, app management, and device diagnostics.
• Hybrid Execution: Combines LLM-driven reasoning for new tasks with deterministic "skills" for repeated workflows.
• On-Device Brain: Supports local LLM execution on the phone via llama.cpp, MediaPipe, and MLX.
• Phone Farm Scaling: Includes a bot runner and job queues to manage multiple devices simultaneously.
• Skill Hub: A community registry for discovering and installing pre-made app automation skills.