takahirom/arbigent

AI Agent for testing Android, iOS, and Web apps. Get Started in 5 Minutes. Arbigent's intuitive UI and powerful code interface make it accessible to everyone, while its scenario breakdown feature ensures scalability for even the most complex tasks.

Arbigent (Arbiter‑Agent) – AI‑Agent Testing Framework

What it is – A desktop application (with a CLI wrapper) that lets you write, run, and manage UI‑level tests for mobile, web and TV apps using AI agents. It breaks a high‑level goal (e.g. “complete the tutorial”) into a chain of dependent scenarios, lets non‑programmers create those scenarios in a visual UI, and lets engineers execute them programmatically from YAML files.

Core ideas

Idea	Why it matters
Scenario dependencies	Complex flows (login → search → purchase) are expressed as small, reusable steps, making tests easier to maintain.
Hybrid UI + code workflow	QA engineers can drag‑and‑drop scenarios in the UI; developers can invoke the same YAML from CI pipelines.
Cross‑platform support	Runs on iOS, Android, web and TV (including D‑pad navigation).
AI‑driven interaction	The agent reads a filtered UI tree and optional AI hints embedded in accessibility labels, then decides what to tap/scroll.
Model flexibility	Defaults to `gpt‑4.1` but can be switched to cheaper models such as `gpt‑4o‑mini`.
Image‑assertion & stuck‑screen detection	Uses Roborazzi’s AI‑powered image assertions to verify that the agent’s actions are correct and can recover when it gets stuck.
MCP (Model Context Protocol) support	External tools (e.g., install apps, fetch logs) can be called from a scenario via JSON‑configured servers.
Reusable scenarios	Library‑style “functions” (similar to GitHub Actions) that can be called with parameters, reducing duplication.
Maestro YAML integration	Existing Maestro test flows can be run as deterministic setup steps before the AI takes over.

Typical workflow

Connect a device (real or emulator) and enter your AI‑provider API key in the Arbigent UI.
Create a scenario – give a natural‑language goal or import an existing Maestro YAML / Android Journeys file.
Add optional hooks – custom init/cleanup code, MCP server calls, or reusable scenario calls.
Run – either click Run in the UI or invoke the CLI (arbigent run --project-file myproj.yml).
Review – the UI shows the UI‑tree, screenshots, and AI‑generated actions; failures can be inspected via image assertions.

Installation

Platform	Steps
macOS (binary)	Download the UI binary from the Releases page; if macOS blocks it, follow Apple’s “Open Anyway” instructions.
CLI	`brew tap takahirom/homebrew-repo && brew install takahirom/repo/arbigent` (requires Java 17+).
Wrapper script	Generate a pinned‑version wrapper (`arbigent wrapper`) that downloads the correct release on first run – useful for CI without a global install.

Who should use it

Role	Benefit
QA engineers	No coding needed to author UI tests; can maintain tests in natural language.
Developers / DevOps	Programmatic execution from CI, parallel sharding, AI‑result caching, and reuse of existing Maestro/YAML assets.
Product teams	Quickly prototype end‑to‑end flows on real devices without writing low‑level UI scripts.

Strengths & Weaknesses (as the author frames them)

Aspect	Rating (1‑5)	Comments
Speed	1	Limited by LLM latency and real‑device interaction; can be mitigated with `--shard` parallelism and result caching.
Maintainability	4	Natural‑language goals and scenario decomposition survive UI tweaks; reusable scenarios cut duplication.
Utilization (cost)	1	Requires device resources and paid LLM calls (≈ $0.005/step, $0.02/task with GPT‑4o). Replay‑with‑fallback can cut recurring AI cost.
Reliability	3	Built‑in waits, dialog handling, and retry help, but emulator flakiness still affects runs.
Fidelity	5	Tests run on real/emulated devices, can verify visual aspects like video playback.

License & Community

Open‑source – free to use, modify, and distribute. The repository invites contributions and provides a specification file for reusable scenarios.
Support – The author notes a spam account impersonating the project; official updates come from the X accounts @_takahirom_ and @new_runnable.

Quick start example (CLI)

# Install the CLI (Homebrew)
brew tap takahirom/homebrew-repo
brew install takahirom/repo/arbigent

# Run a saved project file
arbigent run --project-file myproject.yml \
    --scenario-ids="login-and-play"

Bottom line – Arbigent is a purpose‑built framework that lets teams test modern mobile/web/TV applications with LLM‑driven agents, offering a blend of visual scenario authoring and code‑first execution, while exposing hooks for custom tooling and existing deterministic test assets.