OpSafari/hypoarena

What it solves

It provides a fully offline workbench for testing and developing the underlying mechanisms of "AI co-scientist" pipelines. Instead of relying on live LLMs, it allows researchers to isolate and test specific components—such as hypothesis generation, evidence verification, and ranking—using synthetic data to ensure the system's logic is sound before deploying it to real-world scientific discovery.

How it works

The project decomposes the scientific discovery process into a series of testable stages:

• Synthetic Literature: Generates deterministic corpora with planted causal chains and noise to test recovery rates.
• Generate-Debate-Evolve Loop: Uses pluggable agent adapters to propose, critique, and revise hypotheses through evolution operators like mechanism crossover and claim decomposition.
• Verification & Deduplication: Performs span-level grounding checks (citation and entity overlap) and uses TF-IDF or MinHash LSH to remove duplicate hypotheses.
• Ranking & Accumulation: Employs Bradley-Terry/Elo tournaments to rank hypotheses based on rubrics and uses Bayesian updating to accumulate evidence.
• Reporting: Produces byte-identical Markdown and HTML reports of the results without requiring network access.

Who it’s for

Researchers and developers building AI agents for scientific discovery who need a controlled, offline environment to validate the logic and performance of their hypothesis-discovery pipelines.

Highlights

• Fully Offline: Runs without network access, real models, or weight downloads, depending primarily on NumPy.
• Deterministic Artifacts: Ensures byte-identical outputs across platforms via a canonical serializer.
• Comprehensive Tooling: Includes a full suite of components from synthetic corpus generation to Bayesian evidence accumulation.
• Grounding Verification: Specifically tests for fabricated citations and drifted numbers using adversarial fixtures.