superwesleyhys-ux/factcircuit

FactCircuit: an auditable claim-and-evidence verification loop with immutable provenance, staged checks, bounded retrieval, and reproducible evaluation.

What it solves

FactCircuit addresses the problem of "one-shot" fact-checking where a system simply returns a true/false label without providing a transparent, auditable trail of evidence. It prevents common verification failures such as ignoring source revisions, confusing syndicated stories for independent sources, and failing to track why a specific search was performed.

How it works

The project implements a replayable "evidence circuit" that treats verification as a multi-stage loop rather than a single prompt. It uses an adapter-driven Python harness to manage the following process:

  1. Immutable Snapshots: Every piece of evidence is versioned and hashed, ensuring that revisions to a URL do not overwrite previous data.
  2. Decomposition: Retrieved documents are broken down into atomic claims (atoms) and analyzed for lineage before being admitted to the evidence graph.
  3. Separation of Concerns: The system distinguishes between whether a snapshot supports a claim (evidence critic) and whether the event actually happened in the world (world critic).
  4. Bounded Loops: The process is bounded by rounds, time, and token budgets. If information is missing, it emits a specific task to fetch or reanalyze data, which then re-enters the loop.
  5. Model Tunnels: It supports multiple inference routes, including a local Codex CLI default, OpenAI API, and Anthropic API.

Who it’s for

Researchers and developers building auditable AI verification systems, specifically those dealing with changing news evidence and the need for high-provenance fact-checking.

Highlights

  • Auditable Traces: Preserves the exact path of verification, including source spans, timestamps, and resource usage.
  • Versioned Evidence: Treats same-URL revisions as distinct identities to prevent data loss during verification.
  • ** highlighted Evaluation Toolkit**: Includes a comprehensive suite of metrics (VP, FR, TR, SR, EN, CA, HFAR) to measure precision, recall, and probability quality.
  • Offline Capability: The core standard-library harness can run entirely offline using hand-authored annotations for testing the evidence loop.
  • Double-Loop Runner: An advanced mode where models select follow-up retrieval tasks and request re-analysis of earlier findings.

Related

  • Project
    truecourse-ai/truecourse

    TrueCourse is an AI-powered platform that automatically converts project documentation into end-to-end tests, ensuring that your product's actual behavior always matches its written specifications.

  • Project
    lotus-data/lotus

    LOTUS is a Python library for large‑scale, LLM‑driven data processing. It introduces *semantic operators* (LLM‑based map, filter, reduce, join, etc.) and an optimizer that batches and cascades model calls to improve accuracy while lowering API costs. The library supports both pure LLM operators and *agentic* operators that run tool‑using agents (e.g., a sandboxed Python REPL) on each data shard, enabling complex tasks like code‑base analysis, research synthesis, failure‑mode mining, document extraction, LLM‑as‑judge evaluation, and RAG. Install via `pip install lotus-ai`; usage revolves around creating a `Corpus`, specifying a task, and invoking `agent` or semantic operators. Backed by Stanford/UC Berkeley research and documented on ReadTheDocs, with community Discord and example notebooks.

  • Project
    getsentry/vitest-evals

    vitest‑evals is a monorepo that adds AI‑aware evaluation to Vitest. It provides harness adapters for various model SDKs, reusable judges (e.g., factuality), a custom reporter that outputs detailed JSON, a local React UI for inspecting runs, and a GitHub Action that posts summaries and can gate PR merges based on pass‑rate or scores.

  • Project
    pgalko/BambooAI

    BambooAI is an LLM-driven data analysis platform that prioritizes transparency and reproducibility by maintaining an inspectable, executable record of every computational step taken during an investigation.

  • Project
    cmusphinx/pocketsphinx

    PocketSphinx is an open‑source, offline speech‑to‑text engine (C library with Python bindings) that runs efficiently on low‑resource devices. It provides a command‑line tool and APIs for recognizing or aligning audio, outputting results as JSON.