dondai44423/donsetch

Web fetch, search, and crawl for AI agents. Built from scratch in Rust. No keys, no accounts. AGPL v3.

What it solves

DonSeTch provides AI agents with a high-performance, stealthy way to perform web research. It eliminates the need for multiple API keys and accounts by consolidating fetching, searching, crawling, and screenshotting into a single local process that can bypass bot-detection walls and handle complex web content like PDFs.

How it works

Built in Rust, the project implements its own HTTP/2 transport and extraction engine to remain fast and lightweight. It uses a multi-tier approach to web access: it starts with plain HTTP for speed and automatically escalates to a headless browser (using real Chrome TLS/BoringSSL) to solve challenges and bypass bot walls. Once a challenge is solved, it hands cookies back to the high-speed fetch tier. It also includes a keyless search aggregator that fuses results from over 10 backends with semantic reranking.

Who it’s for

It is designed for developers building AI agents that require reliable web access. It integrates natively with MCP (Model Context Protocol) clients like Claude Code, Cursor, and OpenCode, and also functions as a standalone CLI tool.

Highlights

  • Real Chrome TLS: Uses native BoringSSL to ensure the ClientHello matches a real Chrome browser, avoiding bot detection.
  • Keyless Search: Aggregates and reranks results from 10+ search engines without requiring API keys.
  • Token Efficiency: Uses specialized rendering (e.g., shortening links) and tools like focus and toc to reduce the number of tokens sent to the LLM.
  • Pixel-fusion PDF: Combines glyphs and rendered pixels for deterministic PDF parsing, with auto-OCR for scanned documents.
  • Self-Improving Fetch: Adapts cookie lifetimes and manages cooldowns for domains that block access.
  • Comprehensive Toolset: Provides four primary tools: web_fetch (markdown extraction), web_search (multi-engine search), web_crawl (domain crawling), and web_screenshot (PNG rendering).

Related

  • Project
    1broseidon/ketch

    A stateless CLI tool that unifies web search, code search, and documentation scraping into a single interface designed for both human researchers and AI agents.

  • Project
    vercel-labs/agent-browser

    A high-performance Rust-based browser automation CLI designed to give AI agents reliable, semantic control over web pages and browser state.

  • Project
    rtk-ai/rtk

    A high-performance CLI proxy that compresses bash output for AI agents, reducing input tokens by up to 90% to lower costs and save context space.

  • Project
    Aas-ee/open-webSearch

    Open‑WebSearch is a Node‑JS service that lets AI agents perform live web searches and fetch article content without any API keys. It offers an MCP‑compatible server, a CLI, and a long‑running HTTP daemon, aggregates results from many public search engines, supports proxy use, and can optionally use Playwright‑driven headless browsers for sites that need JavaScript rendering.

  • Project
    lexmount/moli

    Moli is an open‑source, Rust‑based headless browser built for AI agents. It provides a full web runtime (HTML, CSS, JavaScript, networking, storage) but keeps memory and CPU low by performing layout and painting only on demand. The same binary can be used via a simple CLI, Chrome DevTools Protocol, or WebDriver Classic/BiDi, letting existing automation tools (Playwright, Selenium, etc.) connect without extra adapters. Benchmarks show Moli uses ~15 % of Chrome’s CPU time and ~13 % of its memory while achieving comparable success rates on real‑world web tasks. It is production‑ready, cross‑platform, and dual‑licensed (Apache‑2.0 or MIT).