MLGroupJLU/LLM-eval-survey

What it is:
A curated, living collection of research papers and resources about how to evaluate large language models (LLMs) — the AI systems behind chatbots like ChatGPT. It's meant to accompany a formal survey paper (linked in the README) and is updated more frequently than the paper itself.

What it contains:
The repository organizes papers into categories that answer two big questions: what to evaluate and where to evaluate. The “what” section covers areas like natural language processing (sentiment analysis, reasoning, translation, question answering), robustness and bias, social science, medicine, and more. Each entry links to the original paper. The “where” section is listed in the table of contents but not detailed in the README excerpt.

Who made it:
A team of researchers from universities (Jilin, Peking, Westlake, etc.) and Microsoft Research. The README lists authors and affiliations.

How to use it:
You can browse the categorized paper lists, follow the news updates for new versions, and even contribute by submitting pull requests or issues. The project also points to related tools like PromptBench for robustness testing.

Key point:
It's a reference hub — not a software tool — for anyone wanting to understand or conduct LLM evaluation, with a focus on keeping the list current beyond the static arXiv paper.