nicobailon/pi-web-access
Web search and content extraction extension for Pi coding agent
What it solves
Pi Web Access provides a comprehensive web-interaction layer for the Pi agent, enabling it to perform web searches, extract content from various sources, and understand video content. It solves the problem of giving an AI agent a reliable, multi-provider search and content retrieval system with smart fallbacks and specialized handling for complex formats like GitHub repositories, YouTube videos, and PDFs.
How it works
The project implements a set of tools (web_search, fetch_content, get_search_content, and source_check) that the Pi agent can use to interact with the web. It supports a vast array of search providers (e.g., OpenAI, Brave, Exa, Tavily, Gemini, Mistral) and content extraction methods.
Key mechanisms include:
- Smart Fallbacks: A sequential chain of providers is used to ensure search and fetch requests succeed even if one provider fails.
- Specialized Extraction: Instead of simple scraping, it clones GitHub repositories locally for full file access and uses Gemini for visual and transcript-based video understanding.
- PDF Processing: It converts PDFs to Markdown using a tiered system (Datalab, Gemini, or local
unpdf) to maintain document structure. - Caching: Fetched content is stored in a local cache to allow the agent to retrieve specific passages without reloading the entire page into the context window.
Who it’s for
This tool is designed for users of the Pi agent who want to extend its capabilities with real-time web access, deep technical research (via GitHub cloning), and multimodal analysis of video and PDF documents.
Highlights
- Extensive Provider Support: Integrates with dozens of search APIs and supports zero-config search via Exa MCP.
- Video Understanding: Extracts transcripts, visual descriptions, and specific frames from YouTube and local videos using Gemini and ffmpeg.
- get_search_content: Allows the agent to efficiently navigate large pages using offsets and fuzzy text searching.
- GitHub Integration: Clones repositories locally for genuine file exploration rather than scraping HTML.
- Source Checking: A dedicated
source_checktool for verifying claims with machine-readable citations and SHA-256 hashes.
Related
- Project
- Project
- Project
- Project
- Project