oomol-lab/spinedigest

distill any book down to its spine

What it solves

Wiki Graph addresses the challenge of managing long-text knowledge bases for LLMs. Instead of forcing an AI to process raw source material from scratch for every query, it compiles long text into a durable, maintainable, and traceable knowledge base. This prevents the need for repeated extraction and reduces hallucinations by grounding knowledge in public entities and verifiable source evidence.

How it works

It uses a CLI to process long text into .wikg archives. The system extracts entities and relations (triples) from the text, primarily by aligning them with public entities from Wikipedia and Wikidata via WikiSpine. This ensures stability by using a shared public dictionary rather than relying on a private, evolving schema.

Key components include:

  • Knowledge Graph: A network of entities and triples (subject --predicate--> object) that allows users to navigate related concepts.
  • Evidence Tracking: Every entity and relation is linked back to specific chapters and original sentences in the source text for verification.
  • Reading Graph: For summaries, the tool breaks text into cognitive chunks and connects them by relevance to generate compressed summaries that remain traceable to the source.
  • URI System: A stable handle system (e.g., wikg://archive/entity/QID) used to address and operate on specific scopes or objects within the knowledge base.

Who it’s for

It is designed for developers building AI agent workflows that need to ingest large volumes of text (PDFs, EPUBs, web pages, etc.) and turn them into structured, searchable, and verifiable knowledge bases.

Highlights

  • Source Traceability: Every piece of extracted knowledge can be traced back to the original source sentence.
  • Public Entity Grounding: Uses Wikidata/Wikipedia QIDs to ensure consistent entity identification across different documents.
  • Portable Archives: Stores everything in .wikg files that can be shared, backed up, or moved.
  • Structured Navigation: Allows querying the knowledge base via a graph of entities and relations rather than just keyword search.