agent-device: what it is, what problem it solves & why it's gaining traction

What it solves

agent-device provides a way for AI coding and QA agents to interact with and verify real applications across multiple platforms. It bridges the gap between an agent reasoning about code and actually seeing if that code works on a physical device, simulator, or emulator by providing a standardized CLI for device automation.

How it works

The tool acts as the "hands and eyes" for an AI agent. It uses platform-specific backends (XCTest for iOS/tvOS, ADB for Android, AT-SPI for Linux, and a local helper for macOS) to execute commands. The agent uses the CLI to take "snapshots" of the UI—which are structured accessibility data that assign reference IDs (like @e1) to elements—and then performs actions like tapping, typing, or scrolling using those references.

Who it’s for

• AI Agent Developers: Those building coding agents (e.g., using Cursor, Claude Code, or Windsurf) that need a real-app feedback loop.
• QA Engineers: Developers creating AI-driven QA harnesses to automate mobile and desktop app verification.
• App Developers: Developers working with native iOS/Android, Expo, Flutter, or React Native apps who want to integrate AI agents into their development and CI/CD pipelines.

Highlights

• Multi-platform support: Works across iOS, Android, tvOS, Android TV, macOS, Linux, and web.
• Semantic Interaction: Uses accessibility trees to provide token-efficient snapshots and semantic references for reliable element targeting.
• Evidence Collection: Captures screenshots, videos, logs, traces, network traffic, and React render profiles for debugging.
• Replayability: Allows recording interactions into .ad scripts for repeatable e2e checks or export to Maestro YAML.
• Crossparameter Compatibility: Supports native apps as well as frameworks like Expo, Flutter, and React Native.