vakra-dev/reader

Open source web infrastructure for AI. Scrape, crawl, and automate the web, clean markdown, browser sessions, ready for your agents.

What it solves

Reader is a production-grade web scraping engine designed to simplify web access for AI agents. It eliminates the complexity of managing browser instances, bypassing anti-bot protections (like Cloudflare), and cleaning HTML into a format suitable for LLMs.

How it works

Built on top of Playwright, Reader provides three primary primitives: scraping URLs to clean markdown, crawling websites for page discovery, and launching stealthed browser sessions. It handles the "hard stuff" under the hood, including browser pooling, TLS fingerprinting, navigator spoofing, and tiered proxy rotation (standard and premium) to avoid detection.

Who it’s for

Developers building AI agents or LLM-powered applications that require reliable, high-volume web data extraction without fighting bot detection or managing complex browser infrastructure.

Highlights

  • Anti-Bot Stealth: Built-in TLS fingerprinting, WebRTC masking, and navigator spoofing to bypass detection.
  • LLM-Ready Output: Automatically extracts main content and converts pages to clean Markdown or HTML, removing navbars, footers, and popups.
  • Flexible Control: Offers a high-level API for simple scraping and a low-level browser() method that provides a CDP WebSocket for full Playwright/Puppeteer control.
  • Infrastructure Management: Includes a browser pool with auto-recycling, health monitoring, and concurrent processing capabilities.
  • Hybrid Deployment: Available as a cloud API or a self-hosted engine.