knowhere: a document memory layer that transforms unstructured data into hierarchical graphs for agentic RAG

What it solves

Knowhere addresses the problem of "dirty" unstructured documents (like complex PDFs, Office files, and images) that are difficult for AI agents to use. Traditional RAG often flattens these documents into isolated text snippets, causing a loss of semantic context and hierarchy, which leads to lower accuracy and reliability in AI agent responses.

How it works

Knowhere acts as a memory layer that processes documents in two primary stages:

1. Parse and Build Memory: It uses specialized parsers (such as MinerU) to ingest multi-modal files. A proprietary tree-like algorithm then reconstructs the document's original hierarchy and builds a lightweight memory graph, ensuring chunks retain their semantic context and relationships.
2. Agentic Retrieval: Instead of simple vector lookups, AI agents navigate the reconstructed section trees and graph links. This allows agents to "drill down" into relevant regions of a document, similar to how a human reader would, and provide citations based on traceable source paths.

Who it’s for

It is designed for developers building AI agents and Agentic RAG workflows who need to process complex, long-form, or multi-modal documents while maintaining high retrieval accuracy and evidence-based traceability.

Highlights

• Multi-modal Support: Extracts and summarizes tables and images using VLMs, linking them back to source text.
• Hierarchy Preservation: Reconstructs document structures to prevent semantic fragmentation.
• Agentic Navigation: Combines traditional search (RRF) with autonomous navigation of document trees and graphs.
• Broad Format Support: Handles PDF, DOCX, PPTX, XLSX, CSV, images, Markdown, and text.
• Model Agnostic: Supports various LLM/VLM providers including DeepSeek, Qwen, OpenAI, and others.