fivetaku/insane-search

Auto-bypass for blocked websites in Claude Code — Phase 0→3 adaptive scheduler, no API keys

What it solves

It solves the problem of AI agents (specifically Claude Code) being blocked by 403 errors, Web Application Firewalls (WAF), CAPTCHAs, and anti-bot measures when trying to read public web pages. It allows the agent to access content from platforms like X, Reddit, and YouTube without requiring API keys or user logins.

How it works

The tool uses an adaptive escalation pipeline to retrieve content. If a standard fetch fails, it tries the following phases in order:

  1. Special Public Endpoints: Uses platform-specific APIs and feeds.
  2. Lightweight Probes: Tries public API readers, syndication gateways, and alternative URL variants (e.g., mobile or RSS).
  3. TLS Impersonation: Uses curl_cffi to mimic the TLS fingerprints of real browsers (Safari, Chrome, Firefox).
  4. Headless Browser: Launches a real headless browser for pages that require JavaScript rendering.

It also integrates yt-dlp for YouTube captions and metadata, and scans responses for OGP/JSON-LD to extract structured data even from partial pages.

Who it’s for

Developers using Claude Code who need their AI agent to reliably research public web content, social media threads, and video transcripts without manually managing proxies or API credentials.

Highlights

  • Zero Setup: No API keys or signups required; auto-installs dependencies like curl_cffi and yt-dlp on first use.
  • Resilient Retrieval: Escalates through multiple retrieval methods until a public route is found.
  • Broad Platform Support: Works across X, Reddit, YouTube, Hacker News, LinkedIn, Medium, and more.
  • Strict Boundaries: Only accesses public content and explicitly stops at login walls or paywalls.

Related

  • Project
    browserbase/skills

    A collection of skills for Claude Code that enable AI agents to perform browser automation, UI testing, and deep web research via Browserbase.

  • Project
    pedrohcgs/claude-code-my-workflow

    A fork‑ready framework that uses Anthropic Claude Code to automate academic tasks (slides, papers, data analysis, etc.) via specialised agents, a ten‑gate quality suite, and extensible skills; includes setup scripts, pre‑commit hooks, and detailed documentation.

  • Dispatch
    Claude Code Best Practices Guide

    Anthropic provides a comprehensive set of best practices for optimizing Claude Code, an agentic coding environment, focusing on context window management and autonomous verification.

  • Project
    Hao0321/claude-skill-social-post

    A Claude/Codex skill that lets an LLM draft, review, and (optionally) post social‑media comments on Facebook, Instagram and Threads via a controlled Chrome session. It learns a brand’s voice, keeps a cryptographically‑signed ledger of every action, and includes extensive safety checks and test suites. Live posting is off by default; the repo supplies the automation framework, not the private content.

  • Project
    ryfineZ/codex-session-patcher

    A session cleaner and prompt injection tool for AI coding assistants (Codex, Claude Code, OpenCode) designed to bypass refusal responses during security testing and CTF competitions.