Inside Genebench-Pro: Case Studies in Genomic AI Benchmarking

OpenAI has released Genebench-Pro, a rigorous benchmark designed to evaluate the ability of AI models to perform complex genomic analysis. The benchmark tests whether models can synthesize multi-modal genomic data—including long-read sequencing, expression profiles, and pharmacogenomic evidence—to reach clinically or scientifically valid conclusions.

Overview of Genebench-Pro Case Studies

Genebench-Pro utilizes a series of high-complexity case studies that require models to move beyond simple pattern recognition. To succeed, models must perform multi-step reasoning, handle technical noise (such as ambient RNA or mapping artifacts), and apply domain-specific statistical methods like Mendelian randomization or founder reconstruction.

Key Technical Domains and Task Requirements

Clinical and Somatic Oncology

Genebench-Pro evaluates the ability to make treatment decisions based on structural variants. In one case study, models must determine the clinical utility of a synthetic TXR1-directed inhibitor by recovering a target subgroup from a combination of long-read sequencing, expression data, and tumor-quality evidence before interpreting benefit and toxicity.

Functional and Regulatory Genomics

Tasks in this domain focus on distinguishing true biological signals from technical artifacts:

• CRISPR Target Validation: Models must decide if an lncRNA dependency is transcript-specific or driven by nearby-locus effects, requiring them to control for GC toxicity, plate effects, and neighbor-gene repression.
• Chromatin Loop Strength: Models are tasked with quantifying Hi-C loop-strength differences while masking low-mappability contacts and structural-variant artifacts to prevent distortion of the expected-contact model.

Statistical and Population Genetics

These cases test the model's capacity for complex genetic modeling and ancestry inference:

• Drug Target Prioritization: Models must use cis multivariable Mendelian randomization (cis-MVMR) to estimate direct disease effects for proteins in a linked genetic locus, accounting for linkage disequilibrium (LD) and residual local pleiotropy.
• Ancestry and Admixture: Tasks include inferring parent-specific ancestry and admixture timing from phased local-ancestry tracts, which requires repairing reciprocal artifacts and chromosome-specific label inversions.
• Ancient DNA Analysis: Models must infer positive selection from noisy ancient allele-frequency time series by modeling sequencing-error values and placing loci on a consistent derived-allele scale.

Single-Cell and Structural Genomics

These benchmarks focus on data cleaning and calibration:

• eQTL Estimation: Models must estimate genotype effects on activated-monocyte expression, which requires removing ambient RNA and technical contamination before applying the eQTL model.
• Nested Structural Variants: Models must evaluate clinical associations for nested structural subhaplotypes, ensuring that dosage calibration and expression support remain distinct from broader inversion orientations.

Summary of Data Inputs

Across these case studies, models are provided with raw or semi-processed data files, including:

• Registry Covariates: Patient age, sex, tumor burden, and therapy class.
• Genomic Coordinates: Guide IDs, chromosome positions, and GC fractions.
• Association Summaries: SNP positions, effect alleles, beta values, and p-values.
• Cellular Metrics: Per-cell UMI counts for marker genes and contamination markers.
• Ancestry Data: Phased local-ancestry tracts with posterior values and QC annotations.