us/crw

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

What it solves

AI agents often struggle with the complexity of raw HTML and the infrastructure required to scrape websites. fastCRW provides a way to turn any URL into clean, LLM-ready markdown or schema-validated JSON, eliminating the need for developers to manage complex scraping stacks, proxies, or browser rendering.

How it works

Built in Rust, the engine operates as a single static binary that avoids the overhead of heavy runtimes like Node or Chromium in the request path. It offers six core operations: searching the web, scraping single pages, mapping site structures, crawling entire sites, extracting structured data via JSON Schema, and monitoring pages for changes. It can be used as a managed cloud service or self-hosted via Docker.

Who it’s for

  • AI Agent Developers looking for reliable web data via MCP or SDKs (Python, TypeScript).
  • Data Engineers needing high-accuracy web extraction with low latency.
  • Developers wanting to avoid managing infrastructure for proxies and browser rendering.

Highlights

  • High Truth-Recall: Outperforms competitors in capturing actual page content on public datasets.
  • Low Resource Footprint: Uses ~50 MB of RAM at idle and features sub-second cold starts.
  • Agent-Ready: Includes built-in MCP server support and reusable instruction packs for agents.
  • Flexible Deployment: Offers a managed cloud with zero infra or a self-hosted open-source engine.