Spark-To-Paper-Skills/paperjury

PaperJury – AI‑assisted pre‑submission review for LaTeX papers

What it does

• Acts as a Claude Code (or Codex) plugin that reads a research manuscript, generates reviewer‑style comments, and drafts minimal LaTeX patches.
• The workflow follows a strict review → adjudication → edit → re‑check loop. Each comment is first judged by a panel of simulated reviewers; the system then decides whether the issue is:
  1. valid‑fixable – a safe, textual or formatting problem that can be auto‑patched;
  2. author‑required – a substantive gap (missing experiment, evidence, etc.) that must be handled by the author; or
  3. invalid‑drop – a mistaken comment that is discarded.
• Three operating modes are offered:
  • direct‑edit – you ask for a specific change; the tool drafts a patch and waits for your confirmation.
  • review – a mock‑peer‑review is run, producing a list of issues and optional patches.
  • auto – after you give an explicit “auto” policy, the system runs unattended, applying only low‑risk edits automatically and queuing higher‑risk items for you.

Why it matters

• Provides a closed‑loop assistant that does more than raw text generation: it enforces deterministic safety checks (LaTeX compilation, reference consistency, submission‑format compliance) and keeps a machine‑readable ledger of all decisions.
• The authors report that, on a benchmark of 12 held‑out papers, PaperJury improves the macro‑F1 of issue detection from 0.519 (baseline) to 0.656, reduces unsafe edit rate by 4.4×, and finishes each paper in about 2.5 h of compute (≈6.8 M tokens).
• A full “dog‑food” example is shipped with the repo, showing before/after PDFs and a hand‑checked run report.

How to get it

1. Install as a Claude Code skill via the marketplace:

   /plugin marketplace add Spark-To-Paper-Skills/paperjury
   /plugin install paperjury@Spark-To-Paper-Skills
   

   (or clone the repo into ~/.claude/skills/paperjury).
2. Ensure you have Node.js installed; a LaTeX toolchain is optional but enables real compilation checks.
3. Run npm run doctor to verify the environment, then issue a natural‑language command, e.g.

   请审稿，重点检查实验是否足以支持 claim。
   

   The plugin will respond with a problem list, draft patches, and a re‑check report.

What you receive

Output	Content
📋 Problem list	Reviewer‑style issues with evidence, location, and current status.
🧩 Patch	Minimal LaTeX edits for safe issues; high‑risk edits are queued for author review.
🛠️ Re‑check report	Real LaTeX compile results (errors, overfull boxes, missing refs) or a clear fallback if the toolchain is absent.
🧪 Dogfood sample	samples/dogfood/ contains a real 21‑page draft, the original PDF, the revised PDF, and a verified run report.

Who should use it

• Authors with a finished draft who want a quick, systematic sanity‑check before submitting.
• Researchers preparing a final submission and need to verify claim support, formatting, and anonymity compliance.
• Teams that want unattended multi‑round revisions (enable mode: auto and provide a /goal).

Limitations

• It never replaces the author’s scientific judgment; any issue that requires new experiments or domain decisions is returned as author‑required.
• The tool runs locally; only the LLM backend may be cloud‑hosted, depending on your Claude Code environment.
• It currently supports LaTeX, Markdown, and converted Word (.docx) files, but complex tables or equations may lose fidelity during conversion.

Further reading

• Project homepage & interactive overview: https://spark-to-paper-skills.github.io/paperjury
• Full technical paper (arXiv:2606.16322): https://arxiv.org/abs/2606.16322
• Agent guide, design docs, and protocol specifications are in the docs/ and references/ directories of the repository.

---

PaperJury is a genuine open‑source AI‑agent system that helps researchers perform a structured, safety‑guarded pre‑submission review of LaTeX manuscripts.