Ontos-AI/knowhere

What it solves

Knowhere addresses the difficulty of preparing "dirty" or complex unstructured documents (like PDFs and PowerPoint slides) for AI agents. Traditional RAG often relies on flat vector lookups of isolated snippets or brittle OCR/layout extraction, which can lose the document's original structure, reading order, and visual context, leading to unreliable model responses.

How it works

Knowhere transforms documents into a persistent, navigable memory system through a two-step process:

1. Parse and Build Memory: It uses a dual-track approach. The Text Track preserves native structure for reliable text documents, while the Vision Track uses frontier vision models to understand complex pages holistically. Both tracks are normalized into a single hierarchy-native schema that stores navigation trees, linked assets, and cross-document relationships.
2. Agentic Retrieval: Instead of a fixed pipeline, Knowhere provides a corpus of tools (outlines, structural filters, fuzzy recall, etc.). An AI agent then decides how to explore this memory—traversing the section tree or drilling into specific regions—to find and cite traceable evidence.

Who it’s for

It is designed for developers building AI agents and LLM workflows that require high-accuracy retrieval from complex, local, or offline document collections where structural context and visual evidence are critical.

Highlights

• Dual-track Parsing: Combines text-native extraction with vision-based page understanding for complex PDFs and slides.
• Hierarchy-native Memory: Preserves document paths, page ranges, and section nodes rather than disconnected chunks.
• Agent-native Retrieval: Allows agents to autonomously navigate a document's section tree and cross-document graph.
• Page-grounded Citations: Provides traceable references including source documents, section paths, and rendered visual evidence.
• Cross-document Memory Graph: Connects related documents across a namespace using typed entities and keywords.