JordyZomer/lemmalog

Lemmalog – a Datalog‑based memory engine for LLM agents

What it is – Lemmalog is a Rust library (with a CLI and an optional MCP server) that lets an LLM‑driven agent store its observations in a deductive database instead of a plain vector store. Facts are asserted as triples (subject --rel[confidence]→ object) and a Datalog engine derives additional facts, temporal views, aggregations, and canonical entity names. The engine can answer queries, explain why a fact holds, and simulate “what‑if” updates without mutating the real store.

Why Datalog? – The authors argue that an agent’s memory should be verifiable and incrementally updatable. By treating memory as a stratified Datalog program, Lemmalog can:

• keep provenance (which episode produced each fact),
• propagate confidence scores, and
• recompute only the parts of the database that actually changed when new facts arrive.

Key capabilities (all listed as ✅ in the README)

• Full Datalog interpreter with stratification, negation‑as‑absence, and cycle detection.
• Semi‑naïve fix‑point evaluation with delta maintenance for fast incremental updates.
• Bi‑temporal facts (valid_from, valid_to, asserted_at) and a now() built‑in.
• Confidence annotations (product t‑norm) and provenance sets that merge on re‑derivation.
• Proof‑tree generation (why()) with cycle protection.
• Arithmetic in rule heads (e.g., D = Dm + 1) solved linearly.
• Efficient point‑query evaluation via magic‑sets (ask_deep).
• Indexes and WAM‑style backtracking for sub‑microsecond look‑ups on millions of facts.
• Hybrid retrieval for LLM queries: BM25 over rendered facts + entity‑boosting + budget‑aware positional assembly.
• Extraction layer (Extractor trait) with a mock extractor and an LLM‑backed extractor.
• Entity resolution via star‑shaped alias edges, canonical‑view projection, and conflict detection.
• Persistence via snapshot save/load; incremental streaming change feed (Added, Retracted, Cleared).
• “What‑if” simulation that restores the store to a prior byte‑identical state.
• MCP server (JSON‑RPC) so Claude Code or Kimi CLI can call Lemmalog as a tool.
• REPL for interactive rule editing, querying, and debugging.
• Evaluation harness (scenario::run_eval) that measures accuracy, token usage, and latency against the LongMemEval benchmark.

How it is used

1. Observation – an LLM extracts triples from a conversation and sends them to lemmalog_observe (line protocol S --rel[conf]--> O).
2. Rule installation – domain‑specific Datalog rules are installed (e.g., reports_to(X,Y) :- current(X,"manager",Y).).
3. Query – the agent asks a goal (lemmalog_query) and receives bindings; it can also request a proof (lemmalog_why).
4. Context assembly – before answering a user question, AgentMemory::context_for_query builds a token‑budgeted context using BM25 + entity boosting, feeding only the most relevant facts to the LLM.
5. Tool integration – the MCP server exposes the same operations via JSON‑RPC, allowing Claude Code or Kimi CLI to treat Lemmalog as a native tool.

Benchmarks – On the LongMemEval and MemEval suites Lemmalog achieves:

• F1 ≈ 0.49 on the 102‑question MemEval benchmark (≈ 500 K answer‑phase tokens, far fewer than full‑context baselines).
• Competitive scores on the LoCoMo benchmark (F1 ≈ 0.57, second only to PropMem).
• Sub‑millisecond query latency on 4 M‑fact stores.

Maturity – The README marks every listed feature as implemented (✅) except “Leapfrog triejoins (worst‑case‑optimal joins), DBSP streaming deltas” which is still planned. A comprehensive test suite (450 random programs vs. a naive oracle, parser fuzzing, differential testing) is included, and a REPL is provided for manual experimentation.

Who might benefit –

• Researchers building LLM agents that need explainable and incrementally updatable memory.
• Developers of AI assistants who want to replace raw vector stores with a rule‑driven knowledge base.
• Anyone needing deterministic provenance‑aware reasoning over extracted facts (e.g., audit trails, multi‑agent collaboration).

Getting started – Clone the repo, cargo run --bin lemmalog for the REPL, or build the MCP server with cargo build --release --features mcp and register it with Claude Code/Kimi as shown in the README.