songlab-cal/gpn

What it solves

GPN (Genomic Pretrained Network) provides a suite of genomic language models designed to predict the functional effects of genome-wide variants. It addresses the challenge of understanding how mutations in DNA sequences across different species and evolutionary scales provide functional constraints on the genomic sequence.

How it works

The project implements several model families that process genomic data in different ways:

• GPN: A base model that works with unaligned genomes.

• GPN-MSA: A model based on multispecies alignment (MSA) that requires aligned genomes for both training and inference.

• PhyloGPN: A phylogenetic approach that uses alignments during training but does not require them for inference or fine-tuning.

• GPN-Star: The latest iteration which requires aligned genomes for training and inference to predict functional constraints.

These models can be integrated into existing workflows using standard Transformers AutoClasses or via a dedicated command-line interface (CLI) for tasks like calculating logits, embeddings, and variant effect prediction (VEP).

Who it’s for

Researchers in genomics, computational biology, and bioinformatics who need to predict the effects of genetic variants and analyze genomic sequences using pretrained language models.

Highlights

• Multiple Model Architectures: Offers a variety of models (GPN, GPN-MSA, GPN-Star, PhyloGPN) tailored to different data requirements (aligned vs. unaligned).
• Transformers Integration: GPN-Star can be loaded via AutoModelForMaskedLM from the Hugging Face Transformers library.
• Comprehensive Tooling: Includes a CLI for training, variant effect prediction, and embedding extraction.
• Peer-Reviewed Research: Based on multiple publications in journals like Nature and PNAS.