thiswillbeyourgithub/wdoc

Summarize and query from a lot of heterogeneous documents. Any LLM provider, any filetype, advanced RAG, advanced summaries, scriptable, etc

What it solves

wdoc is a Retrieval-Augmented Generation (RAG) system designed to handle large, heterogeneous collections of documents. It solves the problem of querying and summarizing diverse file types—such as PDFs, audio recordings, video lectures, and Anki flashcards—simultaneously, providing sourced answers to avoid hallucinations and detailed summaries that capture the author's reasoning rather than generic takeaways.

How it works

The system uses a multi-stage pipeline to ensure high recall and specificity:

  1. Retrieval: Documents are retrieved via embeddings, often using multi-query retrievers to expand the user's search.
  2. Evaluation: A "weak" LLM (Eve the Evaluator) filters out irrelevant documents from the initial retrieval.
  3. Extraction: A "strong" LLM (Anna the Answerer) extracts relevant information from each remaining document.
  4. Aggregation: A "combiner" LLM (Carl the Combiner) merges these individual answers into a final markdown response. To optimize this, the system uses scipy's hierarchical clustering to batch answers by semantic similarity before combining them.

For summaries, the system splits text into chunks and uses a strong LLM (Sam the Summarizer) to create detailed, logically indented summaries, maintaining context between chunks.

Who it’s for

It is designed for researchers, students, and professionals who deal with extensive and diverse information sources and need a way to get definitive, sourced answers from a large corpus of documents.

Highlights

  • Broad File Support: Supports 15+ file types, including Anki collections, YouTube videos, audio files (with silence removal), PDFs (using 15 different loaders for optimal parsing), and web pages.
  • Flexible LLM Integration: Compatible with 100+ LLMs and various embedding engines via LiteLLM, including local models for privacy.
  • Sourced Answers: Every answer is linked to a unique document hash, allowing users to verify assertions.
  • Detailed Summaries: Focuses on compressing the author's thought process and reasoning rather than providing high-level takeaways.
  • Deployment Options: Available as a CLI tool, a Python library, or a Gradio-based web interface via Docker.
  • Web Search: Includes preliminary support for querying the web via DuckDuckGo.

Related

  • Project
    BBC-Esq/VectorDB-Plugin

    A Windows-based tool for creating a vector database from documents, images, and audio files to enable Retrieval Augmented Generation (RAG) with various LLMs.

  • Project
    rag-web-ui/rag-web-ui

    An intelligent dialogue system that allows users to build custom Q&A services by combining their own document knowledge bases with LLMs using Retrieval-Augmented Generation.

  • Project
    weiwill88/Local_Pdf_Chat_RAG

    A modular Python implementation of a RAG pipeline designed for learning and experimentation, featuring hybrid retrieval and support for multiple LLM backends.

  • Project
    infiniflow/ragflow

    An open-source RAG engine that combines deep document understanding and agentic capabilities to create a high-fidelity context layer for LLMs using complex unstructured data.

  • Project
    OpenBMB/VisRAG

    A vision-language RAG pipeline that embeds documents as images to prevent information loss from text parsing and uses evidence-guided reasoning for multi-image QA.