mishushakov/llm-scraper

What it solves

Extracting structured information from web pages usually requires writing brittle, per-site scraping code that breaks when markup changes. LLM Scraper replaces that with a more flexible approach: it lets an LLM read the page content and output exactly the fields you ask for, defined as a typed schema. This makes it possible to pull out data like news headlines, product details, or search results without hardcoding CSS selectors or XPath expressions.

How it works

You launch a Playwright browser page, point it at any URL, and define a schema (using Zod or JSON Schema) that describes the data you want. LLM Scraper then feeds the page content to an LLM in one of six formats: pre-processed HTML, raw HTML, markdown, readable text (via Readability.js), a screenshot for multimodal models, or custom content from your own function. The LLM rereads the content and returns a JSON object that matches your schema. You can also run a streaming version that yields partial results as they are generated, and a code-generation mode that produces a standalone Playwright script to extract the data without an LLM call afterwards.

Who it's for

Developers building web scraping pipelines who want a more resilient and schema-driven alternative to conventional parsing. It's especially useful for projects that need to normalize unstructured web data into clean records, or for anyone who already works with Playwright and wants to add LLM-powered extraction. Because it supports many LLM providers (OpenAI, Anthropic, Google, Groq, Ollama), it fits teams that already use one of these platforms.

Highlights

• Schema-first extraction using Zod or JSON Schema, with full TypeScript type safety.
• Supports multiple LLM families and providers via Vercel AI SDK.
• Six formatting modes to feed the model — among them HTML, markdown, text, and screenshots.
• Streaming output for partial results.
• Code-generation mode that gives you a reusable Playwright script for extract-in-less-time.
• Built on Playwright, so it works in any modern browser environment.