steel-dev/awesome-web-agents

🔥 A list of tools, frameworks, and resources for building AI web agents

What it solves

Building AI agents that can reliably navigate, scrape, and interact with the web is difficult due to the complexity of modern websites. This project provides a curated directory of tools, frameworks, and resources to help developers overcome these challenges and build effective web-based AI agents.

How it works

This is a comprehensive, categorized collection of open-source and commercial projects. It organizes tools into specific domains, such as autonomous agents that can browse the web, automation frameworks that translate natural language into browser actions, and specialized scrapers that turn web content into LLM-friendly data. It also includes developer-focused infrastructure, benchmarks, and research to support the full lifecycle of agent development.

Who it’s for

This list is designed for developers, researchers, and AI engineers who are building, testing, or researching AI agents capable of performing tasks on the web, such as data extraction, workflow automation, or autonomous research.

Highlights

  • Autonomous Web Agents: A list of agents that can navigate the web and perform tasks with minimal human input.
  • AI Web Automation Tools: Frameworks and libraries for turning natural language instructions into browser actions.
  • Developer Tools: Infrastructure, headless browser APIs, and debugging utilities specifically built for AI agent workflows.
  • AI Web Scrapers/Crawlers: Specialized tools for extracting structured data from websites for use in LLM applications.

Related

  • Project
    WecoAI/aideml

    AIDE ML is an open‑source Python package that implements an LLM‑guided tree‑search agent for automatically writing, debugging, and benchmarking machine‑learning code. Users specify a dataset, a natural‑language goal, and an evaluation metric; the agent iteratively generates and refines Python scripts, pruning the search tree based on metric feedback. The repo includes a CLI, a Streamlit UI, HTML visualisation, Docker and local‑LLM support, and is designed for both research on agent architectures and practical ML pipeline generation.

  • Project
    MeetKai/functionary

    Functionary is a language model specialized in interpreting and executing functions and plugins, supporting parallel tool calls and a built-in code interpreter.

  • Project
    amd/gaia

    GAIA is an open-source agent factory by AMD that enables developers to build and run AI agents locally on Ryzen AI hardware, focusing on privacy, performance, and hardware-accelerated inference.

  • Project
    zcbacxc/mcd-party-planner

    A Python CLI (and WorkBuddy skill) that uses McDonald’s MCP API to generate real‑time group‑meal quotes, book themed party slots, and suggest optimal dates, all with standard‑library code and built‑in rate‑limit handling.