allenai/reward-bench

RewardBench: the first evaluation tool for reward models.

What it solves

RewardBench provides a standardized way to evaluate the capabilities and safety of reward models (RMs), which are used to align LLMs with human preferences. It addresses the lack of consistent benchmarking for these models, including those trained via Direct Preference Optimization (DPO) and generative "LLM-as-a-judge" models.

How it works

The project provides a suite of tools and datasets to test how well a reward model can distinguish between "chosen" and "rejected" responses. It includes:

  • Inference Code: Unified code to run various reward models (e.g., Starling, PairRM, OpenAssistant).
  • Evaluation Scripts: Specialized scripts for standard reward models (run_rm.py), DPO models (run_dpo.py), and the updated RewardBench 2 (run_v2.py).
  • Generative Support: Tools to use LLMs as judges via local vLLM inference or APIs (OpenAI, Anthropic, Google Gemini, Together).
  • Analysis Tools: Utilities for visualizing results, calculating section-specific scores (Chat, Safety, Reasoning), and testing reward model ensembles.

Who it’s for

  • AI Researchers: Developing and testing new reward models or preference optimization techniques.
  • ML Engineers: Evaluating the safety and quality of reward models before deploying them in RLHF pipelines.
  • Model Developers: Comparing their models against a public leaderboard to gauge performance.

Highlights

  • Broad Model Support: Compatible with sequence classifiers, DPO/KTO models, and generative judges.
  • RewardBench 2: Includes advanced data handling for "best-of-4" and "Ties" datasets.
  • Flexible Deployment: Supports installation via uv or pip and provides Docker images for reproducible research.
  • Integration: Allows pushing evaluation results and model metadata directly to the Hugging Face Hub.

Related

  • Project
    MatthewCYM/VoiceBench

    VoiceBench is a benchmark suite for evaluating LLM‑based voice assistants. It supplies a Hugging‑Face dataset with audio‑driven QA and instruction‑following tasks, a command‑line toolkit to run a model, collect spoken responses, and automatically score them (via GPT‑4o‑mini for open‑ended sets). Results are posted on a public leaderboard, and the repo includes a curated list of recent voice‑assistant papers.

  • Project
    stanford-star/relbench

    RelBench is an open‑source benchmark suite for deep learning on relational databases, offering ready‑to‑load real‑world datasets, a unified task API, PyTorch‑Geometric examples, a public leaderboard, and support for users to add their own data via a simple folder format.

  • Project
    run-llama/ParseBench

    ParseBench is an Apache‑2.0 benchmark that evaluates PDF‑parsing tools on five dimensions (tables, charts, content faithfulness, semantic formatting, visual grounding) using a 2 k‑page enterprise dataset. It ships a CLI, 180+ ready‑made pipelines (LlamaParse, OpenAI, Anthropic, Google, Azure, etc.), deterministic rule‑based metrics, and generates interactive HTML reports and leaderboards.

  • Project
    huggingface/optimum-benchmark

    Optimum‑Benchmark is a Hugging Face library for measuring inference and training performance (latency, memory, energy, throughput) of Transformers, Diffusers, PEFT, etc., across many back‑ends (PyTorch, ONNX Runtime, OpenVINO, vLLM, TensorRT‑LLM, IPEX, Llama‑Cpp, Py‑TXI) and devices (CPU, CUDA, ROCm, XPU, HPU). It provides a Python API and a Hydra‑based CLI, supports distributed launches, and outputs reproducible reports that can be pushed to the Hugging  Face Hub.

  • Project
    X-GenGroup/Flow-Factory

    Flow‑Factory is an open‑source Python framework that unifies supervised fine‑tuning and reinforcement‑learning style optimization (GRPO, DPO, DPPO, etc.) for diffusion and flow‑matching generative models across image, video, and audio‑video tasks. It supports many state‑of‑the‑art models (Stable‑Diffusion 3.5, FLUX 1/2, Qwen‑Image, Wan2.x, MiniMax H3, etc.), offers a flexible YAML‑driven training pipeline, integrates with popular experiment trackers, and provides detailed guidance for adding new models or datasets.