yusukebe/ax

What it solves

ax replaces the inefficient loop of fetching, understanding, and extracting data from the web that coding agents typically perform. Instead of relying on curl piped into fragile parsing scripts or dumping massive amounts of raw HTML into a context window, ax provides a single command to fetch pages, discover their structure, and extract structured data in a format optimized for LLM token limits.

How it works

The tool operates as a local, deterministic binary that combines several capabilities:

• Fetching: It provides full reports (status, headers, redirects) for every request, ensuring agents are never left guessing when a body is empty.
• Discovery: It uses flags like --outline to show repeating structures and --locate to find specific selectors without dumping the entire HTML page.
• Extraction: It can pull multi-field rows (--row), convert HTML tables into keyed rows (--table), and filter results using a safe expression language (--where).
• Token Management: It implements a token budget (--budget) and result caps to prevent context window overflow, providing precise offsets for agents to fetch remaining data without overlap.

Who it’s for

It is primarily designed for AI coding agents that need to interact with the web to gather information, read documentation, or scrape structured data without writing and debugging custom Python scripts for every task.

Highlights

• Agent-optimized output: Specifically designed to be token-cheap and structured for LLM context windows.
• Integrated workflow: Combines fetching, structure discovery, and extraction into one tool, eliminating the need for multiple binaries or scripts.
• Local and deterministic: Operates locally without requiring API keys or cloud-based markdown conversion.
• Pagination support: Uses --json-envelope and --offset for machine-readable continuation of large datasets.
• Markdown conversion: Can convert pages directly to readable markdown via the --md flag.