SamurAIGPT/llm-wiki-agent

LLM Wiki Agent – a self‑maintaining knowledge wiki built by LLMs

What it is – A local, file‑based knowledge‑base that lets you drop any kind of document (PDF, DOCX, PPTX, markdown, etc.) into a raw/ folder, then ask an LLM‑powered agent to ingest it. The agent extracts entities, concepts and summaries, creates inter‑linked markdown pages, flags contradictions, and updates a persistent knowledge graph. All output lives as plain markdown and a static graph.html visualisation, so you can browse it with any editor (e.g., Obsidian) or a web browser.

Why it matters – Unlike typical Retrieval‑Augmented Generation (RAG) pipelines that re‑process raw chunks on every query, this tool compiles knowledge once and keeps it up‑to‑date. Cross‑references, entity pages and contradiction flags are generated at ingest time, giving you a growing, query‑ready wiki without the “forget‑after‑session” problem of chat‑only assistants.

---

Quick start (no Python code required)

# clone the repo
git clone https://github.com/SamurAIGPT/llm-wiki-agent.git
cd llm-wiki-agent

# open the repo with any supported LLM‑agent (Claude, Codex, Gemini, etc.)
# the agents read the config files (CLAUDE.md, AGENTS.md, GEMINI.md)
claude      # or `codex`, `gemini`, `opencode`

The agents understand natural‑language commands such as ingest …, query: …, lint, and build graph.

---

Core workflow

Step	Command (natural language)	What happens
Ingest	ingest raw/papers/attention-is-all-you-need.md	The file is converted to markdown (if needed), parsed, and the wiki is updated. New entity pages (people, companies) and concept pages (ideas, methods) are auto‑created.
Query	query: what are the main themes?	The LLM reads the structured wiki, synthesises an answer, and offers to save it as a synthesis page under wiki/syntheses/.
Lint	lint	Runs a consistency check: orphan pages, broken links, missing entity pages, and detected contradictions are reported in wiki/log.md.
Graph	build graph	Generates graph.json (node/edge cache) and a self‑contained graph.html visualisation powered by vis.js.

---

What you get on disk

wiki/
├─ index.md          # auto‑generated catalog of all pages
├─ log.md            # append‑only operation log
├─ overview.md       # living synthesis of all sources
├─ sources/          # one summary page per original document
├─ entities/         # auto‑created pages for people, orgs, projects
├─ concepts/         # auto‑created pages for ideas, frameworks, methods
└─ syntheses/        # answers you asked the agent to keep

graph/
├─ graph.json        # cached node/edge data (SHA‑256 keyed)
└─ graph.html        # interactive browser view of the knowledge graph

All files are plain markdown, so they work with any editor, version‑control system, or Obsidian vault.

---

Highlighted features

• Multi‑format ingest – PDFs, DOCX, PPTX, XLSX, HTML, CSV, JSON, EPUB, etc. are auto‑converted via Microsoft’s markitdown (or optional tools like Marker for complex PDFs).
• Entity & concept auto‑creation – The agent extracts proper nouns and technical terms, builds dedicated pages, and links them everywhere they appear.
• Contradiction detection – When a new source disagrees with an existing claim, a flag is added at ingest time.
• Persistent knowledge graph – Two‑pass build: deterministic wikilinks + LLM‑inferred semantic edges (confidence‑scored). Louvain community detection clusters topics.
• Lint reports – Orphan pages, broken links, data gaps, and suggested missing sources.
• Obsidian‑ready – The wiki uses standard [[wikilink]] syntax; you can symlink wiki/ into any Obsidian vault for a live graph view.
• Zero‑server, zero‑database – Everything lives on the filesystem; the only runtime dependency is the LLM‑agent that reads the config files.

---

Example use cases

Domain	How you would use it
Research	Ingest papers (/wiki-ingest raw/papers/*.md), then ask the agent for synthesis like “What are the main approaches to reducing hallucination?” – the answer is stored as a wiki page for future reference.
Reading a book	Drop each chapter as markdown, let the agent create character and theme pages, then query narrative arcs or contradictions.
Personal knowledge base	File journal entries, article summaries, podcast notes; query patterns (e.g., “What habits improve my energy?”) and watch the wiki evolve.
Team intelligence	Ingest meeting transcripts, roadmaps, customer calls; ask “What feature requests appear most often?” and get a structured answer plus a flag if the roadmap contradicts customer feedback.
Competitive analysis	Continuously feed competitor announcements and market reports; query comparative tables or generate a “competitive landscape” synthesis page.

---

Tech stack (as listed in the README)

• LLM agents – Claude Code, OpenAI Codex, Gemini CLI (any agent that can read the provided schema files).
• Graph processing – NetworkX + Louvain community detection.
• Visualization – vis.js (static graph.html).
• File conversion – markitdown (default), optional Marker, arxiv2md, PyMuPDF4LLM for PDFs.
• Scripting – Small Python utilities in tools/ for bulk conversion; they only need the above Python packages and an Anthropic API key if you run the conversion scripts directly.

---

Getting help / extending

• The repository ships schema files (CLAUDE.md, AGENTS.md, GEMINI.md) that define the exact command set for each agent. Editing these lets you customise naming conventions, front‑matter fields, or add new triggers.
• Because the wiki is just a git repository, you can version‑track changes, branch for experiments, or merge contributions from collaborators.

---

TL;DR – LLM Wiki Agent is a locally run, agent‑driven pipeline that turns any collection of documents into a living, inter‑linked markdown wiki with automatic entity extraction, contradiction flagging, and a visual knowledge graph. It works with any Claude/Codex/Gemini‑style LLM agent, requires no server, and integrates seamlessly with tools like Obsidian.