block/trailblaze

🥾 AI-Driven UI Testing Framework with Recorded Trails

What it solves

Trailblaze provides a way for AI coding agents (like Claude Code, Cursor, or Goose) to interact with and control real devices across iOS, Android, and the web. It bridges the gap between an agent's ability to write code and its ability to actually execute actions on a user interface, while also turning those agent-driven sessions into deterministic, replayable tests that can be run in CI without needing an LLM at replay time.

How it works

Trailblaze operates as a CLI tool that agents can use to read the screen (via snapshot) and perform actions (via tool). It exposes the native accessibility trees and UI semantics of each platform—Android's accessibility tree, iOS's native UI semantics, and the web's DOM—allowing agents to target elements semantically.

When an agent drives a device, the session is saved as a "trail" (trail.yaml). This file contains natural-language descriptions of the steps and recorded actions. These trails can be replayed deterministically in CI. If a flow breaks due to small UI changes, Trailblaze can use a built-in agent to "self-heal" the test by patching small drifts in the UI against the live screen.

Who it’s for

  • AI Agent Developers: Those building or using coding agents to automate UI testing and device control.
  • QA Engineers: Engineers who want to turn ad-hoc AI-driven exploration into committed, version-controlled regression tests.
  • App Developers: Developers building cross-platform apps who need a native-fidelity automation tool that doesn't limit functionality to the lowest common denominator.

Highlights

  • Cross-Platform Control: Unified CLI for iOS, Android, and Web.
  • Deterministic Replay: Convert AI sessions into trail.yaml files for CI-ready tests that run without an LLM.
  • Coded Custom Tools: Ability to define typed custom commands (e.g., login or addToCart) in TypeScript for more complex agent interactions.
  • Agent Skills: Ships with a portable SKILL.md format to quickly teach coding agents how to use the CLI.
  • Trace Viewer: A browser-based report viewer providing screenshots, view hierarchies, and video for every step.
  • Self-Healing Tests: Optional --self-heal mode to use an agent to fix minor UI drift during replay.

Related

  • Project
    firecrawl/cli

    Firecrawl CLI is a terminal tool that lets you search, scrape, and interact with the Firecrawl AI‑powered web service. It supports commands for raw scraping, web search with highlights, developer‑code index queries, and a massive research‑paper index. The CLI can install “skills” that teach supported AI coding agents (Claude Code, Cursor, OpenHands, etc.) how to use Firecrawl, and it can be scoped to specific agents. Authentication is handled via browser login or API key, and self‑hosted instances are supported via a custom API URL. Install with npm (`npm install -g firecrawl-cli`) or a one‑shot `npx` init command.

  • Project
    droidrun/mobilerun

    An open-source framework that enables LLM agents to control Android and iOS devices using natural language, providing tools for UI inspection and mobile automation.

  • Project
    droid-ash/finalrun-agent

    An AI-driven CLI that tests Android and iOS apps using natural language specifications, utilizing multimodal models to interact with the UI and generate test reports.

  • Project
    lycorp-jp/sim-use

    A cross-platform CLI that enables AI agents to observe and act on iOS and Android screens via token-efficient UI outlines and simplified interaction commands.

  • Project
    microsoft/playwright

    Playwright is a Microsoft‑maintained open‑source framework for automating Chromium, Firefox and WebKit. It offers a full‑featured test runner (Playwright Test), a low‑level library for scripts, a command‑line tool for coding‑assistant agents, and an MCP server that exposes a structured accessibility tree so LLMs can control browsers deterministically. Available for Node.js, Python, .NET and Java, it runs on all major OSes and integrates with VS Code for debugging and trace viewing.