MODSetter/SurfSense

Open-source NotebookLM alternative. Research the open web with live data(Reddit, YT, IG, TikTok, Indeed, Google Search, Maps etc) through one platform, API or MCP server. Join our Discord: https://discord.gg/ejRNvftDp9

What it solves

SurfSense provides a way for AI agents to access live, structured data from the open web without the fragility of custom scrapers or the high cost and latency of browser-based agents. It replaces the need for complex HTML parsing and rate-limit management when gathering information from platforms like Reddit, YouTube, and Amazon.

How it works

SurfSense operates as a research platform that exposes various web connectors as a single REST API or an MCP (Model Context Protocol) server. This allows AI agents to call specific tools (e.g., surfsense_reddit_scrape) to receive structured JSON data instead of raw HTML. It also includes a research workspace with a knowledge base for storing findings, a deliverable studio for creating reports and podcasts, and an automation engine for scheduled research tasks.

Who it’s for

It is designed for developers building AI agents, researchers who need a structured way to monitor the web, and users seeking an open-source, self-hostable alternative to Google NotebookLM.

Highlights

  • Live Data Connectors: Built-in scrapers for Reddit, YouTube, Instagram, TikTok, Google Maps, Google Search, Indeed, and Amazon.
  • MCP Server Support: Native tool integration for agents using Claude, Cursor, or other agent frameworks.
  • Knowledge Base: Hybrid semantic and full-text search for uploaded documents and synced cloud storage (Google Drive, OneDrive, Dropbox).
  • Deliverable Studio: AI-generated reports, editable slide decks, and two-host AI podcasts.
  • Extensible LLM Support: Compatible with 100+ LLMs via LiteLLM, including local models via Ollama and vLLM.
  • Self-Hostable: Can be deployed via Docker for full data privacy and free usage.

Related

  • Project
    firecrawl/firecrawl-mcp-server

    An MCP server that brings Firecrawl's web scraping, searching, and browser interaction capabilities to AI agents for clean, structured web context.

  • Project
    brightdata/brightdata-mcp

    An MCP server that gives AI agents real-time web access with built-in bot and CAPTCHA bypassing, offering 69 tools for search, scraping, and structured data extraction.

  • Project
    firecrawl/firecrawl

    An API for searching, scraping, and interacting with the web at scale, converting website content into clean Markdown or structured data for AI agents.

  • Project
    unclecode/crawl4ai

    An open-source web crawler and scraper that converts websites into clean, LLM-ready Markdown for RAG, AI agents, and data pipelines.

  • Project
    Anakin-Inc/anakin

    An open-source web scraping API that converts websites into LLM-ready markdown or structured JSON, featuring an anti-detect browser and ML-powered proxy selection.