NeuroDong/Ai-Review

Large model-assisted paper review

🤖 What is Ai‑Review?

Ai‑Review is an open‑source web service that lets researchers get an automated, AI‑generated review of a scientific manuscript. By uploading a PDF (or LaTeX/Word source) the system produces a structured list of Strengths, Weaknesses, and Suggestions and can also tell you how competitive your paper is compared to other works from the same venue.

🔑 Core capabilities

Feature What it does How you access it
LLM‑based review Generates a textual review using large‑language‑model prompting (SoT, few‑shot, chain‑of‑thought). Web UI – https://ai-review.neurodong.top
VLM‑based review Takes image snapshots of each PDF page so the model can see figures, tables, and layout, then adds image‑quality feedback. …/vlm_review.html
Prompt‑injection detection Scans the PDF for hidden instructions that could hijack the reviewer model. …/prompt_injection.html
Relative Rank Places your anonymized manuscript in a pool of papers from a chosen conference (e.g., ICLR, NeurIPS) or a custom set, and returns a competitiveness ranking. …/compete.html
Agent‑Skill integration Exposes the review functionality as a Skill for AI‑assistant platforms (e.g., Cursor). You can trigger it with natural language like review my paper or Chinese commands 审稿. Install ai-review-skills folder under the platform’s skill directory

🛠️ How it works under the hood

  • LLM back‑end – The service can call any compatible LLM API (default demo uses a public endpoint; you can configure your own, e.g., Deepseek). Prompt templates live in Prompts/ and include reverse‑prompt, few‑shot, and chain‑of‑thought variants.
  • VLM back‑end – Uses Facebook‑research’s Nougat‑style visual‑language model to extract page snapshots and reason about figures and layout.
  • PDF processing – For LLM reviews the text is extracted; for VLM reviews the images are captured and fed to the VLM.
  • Ranking engine – Text from all PDFs in a pool is anonymized, embedded (presumably with the same LLM), and compared pair‑wise; the reported pairwise accuracy on 100 test pools is 83.8 %.
  • Deployment – Hosted on Cloudflare (static front‑end + serverless functions) with a Gradio‑style UI; the codebase also includes a FastAPI back‑end for local deployment.

🚀 Getting started

  1. Web demo (no install) – Open the website, drag‑and‑drop your PDF, pick a prompting mode, and click Review.
  2. Run locally – Clone the repo, install the Python dependencies (FastAPI, Gradio, the VLM model), and start the server (uvicorn app:app).
  3. Add as a Skill – Copy the ai-review-skills folder to ~/.cursor/skills/ (or the equivalent for your assistant) and invoke it from chat.

📚 Resources in the repo

  • Prompts/ – Prompt templates used for LLM and VLM reviews.
  • Examples/ – Sample reviews (e.g., ResNet paper) and the Relative‑Rank evaluation report.
  • ai-review-skills/SKILL.md – Instructions for installing the Agent Skill.
  • Examples/Relative_Rank_evaluation.md – Detailed accuracy numbers for the ranking feature.

🗓️ Recent updates (as of Aug 2026)

  • 20 Aug 2026 – Launched Relative Rank with an evaluation of 100 pools (83.8 % pairwise accuracy).
  • 19 Mar 2026 – Added prompt‑injection detection.
  • 06 Mar 2026 – Integrated layout awareness and image‑aesthetic scoring into VLM review.
  • 26 Feb 2026 – Released the Cursor‑compatible Ai‑Review Skill.
  • Earlier 2025 – Added chain‑of‑thought, few‑shot, and side‑by‑side prompt comparison modes; moved hosting to Cloudflare.

🎯 Who might use this?

  • Researchers preparing a conference submission who want a quick sanity‑check of their manuscript.
  • Graduate students looking for structured feedback before sharing drafts with advisors.
  • AI‑assistant developers who want to embed paper‑review capabilities into their bots.

📌 Bottom line

Ai‑Review is a fully functional, continuously updated platform that combines LLM text analysis, VLM visual reasoning, and a competitive ranking engine to help scientists improve their papers. It can be used instantly via the public website or integrated into personal workflows through a FastAPI/Gradio server or as an Agent Skill.

Related

  • Project
    Tswoen/Paper-Agent

    Paper‑Agent is an open‑source AI research assistant that automates literature review creation. It uses a LangGraph‑driven workflow of specialized agents (search, read, analyse, outline, write) that call LLMs (OpenAI/Anthropic compatible) to retrieve papers from arXiv, OpenAlex, and Semantic Scholar, download PDFs, convert them to markdown, chunk and embed the text in ChromaDB, analyse sub‑topics, generate a structured outline, and write each section of a review. A FastAPI backend streams real‑time progress via SSE, persisting sessions in SQLite + files, while a Vue 3 front‑end lets users configure model providers, monitor token usage, and resume work after refresh. Installation uses the `uv` Python package manager and npm for the UI; configuration is done via `config/model.json`. The project is fully open source and invites contributions via tests and PRs.

  • Project
    AISBench/benchmark

    A model evaluation tool based on OpenCompass that provides accuracy and performance benchmarking for both local and service-based AI models across text and multimodal tasks.

  • Project
    clusterzx/paperless-ai

    Paperless‑AI is an open‑source extension for the self‑hosted document manager Paperless‑ngx. It automatically classifies new documents, adds AI‑generated titles/tags, and builds a vector index so you can ask natural‑language questions across your whole archive (RAG chat). It works with any OpenAI‑compatible model—including local Ollama servers—and ships as a Docker image with a simple web UI for configuration and manual processing.

  • Project
    wdcpclover/ai4paper

    An AI-powered research workbench that integrates literature search, PDF reading, and management with a grounded AI system for writing professional, traceable literature reviews.

  • Project
    VigoZhao/AI-Visual-Prompt-Cookbook

    A curated library of 116 JSON‑based visual‑prompt templates for AI image generation. Each style defines a reusable prompt with placeholders, fidelity anchors, and negative prompts, plus preview images. Users copy the `style.json` into a multimodal LLM (ChatGPT‑4‑Vision, Gemini‑Pro‑Image, etc.), fill in variables, and generate consistent poster‑style images. Includes validation script, multilingual docs, and an online gallery.