laiso/site2pdf

Generate comprehensive PDFs of entire websites, ideal for RAG.

What it solves

It simplifies the process of converting an entire website into a single, portable PDF document. This is specifically designed to make it easier to feed website content into AI tools like Google NotebookLM or ChatGPT GPTs for Retrieval-Augmented Generation (RAG) and Question Answering (QA) tasks.

How it works

The tool uses Puppeteer to launch a headless Chrome browser, crawls a starting URL to find matching links based on a a provided pattern, and generates PDFs for each page concurrently. Finally, it merges these individual PDFs into one comprehensive document using the pdf-lib library.

Who it’s for

Developers and AI practitioners who need to convert web-based documentation or sites into a format that is easily uploadable to LLM-based knowledge bases.

Highlights

  • AI-Ready: Specifically optimized for RAG and multimodal AI models by preserving visual formatting and images.
  • Concurrent Processing: Speeds up the generation process by processing multiple pages in parallel.
  • Flexible Filtering: Allows users to specify regex patterns to include only specific sections of a website.
  • Portable Output: Consolidates multiple web pages into one shareable PDF file.

Related

  • Project
  • Project
  • Project
  • Project
  • Project