tongjingqi/AI-Can-Learn-Scientific-Taste

We propose Reinforcement Learning from Community Feedback (RLCF), a training paradigm that uses large-scale community signals as supervision, and formulate scientific taste learning as a preference modeling and alignment problem.

What it solves

It addresses the challenge of teaching AI "scientific taste"—the ability to judge the potential long-term impact of research ideas and propose new ones that are likely to be influential. Traditionally, this expert foresight is a human-centric skill that is difficult to quantify or automate.

How it works

The project uses a training paradigm called Reinforcement Learning from Community Feedback (RLCF) in three stages:

  1. Community Preference Construction: It converts citation data from arXiv into pairwise preference signals by matching papers from the same field and time period.
  2. Scientific Judge: A generative reward model is trained using Group Relative Policy Optimization (GRPO) on these pairs to reason over abstracts and predict which paper has higher potential impact.
  3. Scientific Thinker: A policy model is optimized using the Scientific Judge as a reward model (via comparison-based GRPO) to generate follow-up research ideas based on a given paper's title and abstract.

Who it’s for

This is for AI researchers and scientists interested in automated scientific discovery, reward modeling for specialized domains, and the application of RLHF/RLCF to academic research.

Highlights

  • High Accuracy: The 30B Scientific Judge variant achieved 82.7% in-domain accuracy, outperforming baselines like GPT-5.4 Thinking.
  • Temporal Generalization: The model maintains high performance when judging papers from future years (e.g., 2025 papers).
  • Cross-Domain Transfer: The learned judgment generalizes across different scientific fields (including biology via bioRxiv) and different community metrics like ICLR peer reviews.
  • SciJudgeBench: Includes a large-scale dataset of over 720,000 preference pairs across Computer Science, Mathematics, and Physics.

Related