Tencent-Hunyuan/CL-bench

What it solves

CL-bench addresses the challenge of evaluating how well language models can learn new, unseen information directly from a provided context at inference time. It moves beyond static knowledge retrieval to measure a model's ability to synthesize and apply new rules, procedures, and domain-specific knowledge provided in the prompt.

How it works

The project provides two datasets, CL-bench and CL-bench Life, consisting of expert-annotated tasks. Each task includes a system prompt, a context containing new information, and a set of rubrics. The evaluation pipeline runs the model on these tasks and uses a judge model (e.g., GPT-5.1) to verify if the output satisfies all defined rubrics. A task is only considered solved if the model passes every rubric associated with it.

Who it’s for

This is for AI researchers and developers who need to benchmark the reasoning and in-context learning capabilities of large language models against complex, real-world, and domain-specific scenarios.

Highlights

• Comprehensive Evaluation: Includes 1,899 professional/domain-specific tasks and 405 real-life scenarios like group chats and activity logs.
• Expert-Annotated: Every task is verified by domain experts, with an average of 63.2 rubrics per context in the professional benchmark.
• Standardized Pipeline: Provides ready-to-use Python scripts for inference and evaluation, supporting both OpenAI and custom API endpoints.
• Transparent Scoring: Uses a strict binary evaluation method where models must satisfy all rubrics to succeed, ensuring high-quality performance measurement.