OpenAI and Retro Biosciences: Accelerating Life Sciences with GPT-4b micro

OpenAI and Retro Biosciences have developed GPT-4b micro, a specialized miniature version of GPT-4o designed for protein engineering. In collaboration with Retro Biosciences, the model was used to re-engineer proteins involved in stem cell reprogramming, resulting in redesigned proteins that achieved greater than a 50-fold higher expression of stem cell reprogramming markers than wild-type controls and demonstrated enhanced DNA damage repair capabilities.

GPT-4b micro: A Specialized Model for Protein Engineering

GPT-4b micro is a custom model initialized from a scaled-down version of GPT-4o and further trained on a dataset consisting of protein sequences, biological text, and tokenized 3D structure data. This architecture allows the model to handle both structured proteins and those with intrinsically disordered regions, such as the Yamanaka factors.

Technical Training and Data Enrichment

To improve steerability and flexibility, the training data was enriched with:

  • Contextual Information: Textual descriptions of proteins.
  • Co-evolutionary Homologous Sequences: Data on related protein sequences.
  • Interaction Groups: Information on proteins known to interact.

This enrichment increased the effective context length of training examples. During inference, the model supports prompts as large as 64,000 tokens, a context size described as unprecedented for protein sequence models.

Scaling Laws and Validation

OpenAI observed scaling laws similar to those in language models, where larger models and datasets yielded predictable gains in perplexity and downstream benchmarks. However, because in silico evaluations are often insufficient for real-world utility, the model's capabilities were validated through wet-lab experiments in partnership with Retro Biosciences.

AI-Assisted Re-engineering of Yamanaka Factors

The Yamanaka factors (OCT4, SOX2, KLF4, and MYC, or OSKM) are essential for reprogramming adult cells into pluripotent stem cells. However, standard OSKM reprogramming is typically inefficient, with less than 0.1% of cells converting and the process taking three weeks or more.

Overcoming the Design Space Challenge

Traditional "directed-evolution" screens are limited because the number of possible variants for proteins like SOX2 (317 amino acids) and KLF4 (513 amino acids) is approximately 10^1000. Previous academic efforts involving thousands of mutants or 15 years of chimeric protein research have yielded only modest gains.

Results of the RetroSOX and RetroKLF Screens

Using a wet-lab screening platform with human fibroblast cells, Retro Biosciences used GPT-4b micro to propose "RetroSOX" sequences. The results showed:

  • High Hit Rate: Over 30% of the model's suggestions outperformed wild-type SOX2 in expressing key pluripotency markers, compared to typical hit rates below 10% in traditional screens.
  • Deep Sequence Edits: The successful variants differed by more than 100 amino acids on average from the wild-type.
  • Accelerated Reprogramming: Combining top RetroSOX and RetroKLF variants led to a dramatic rise in early (SSEA-4) and late (TRA-1-60, NANOG) markers, with late markers appearing several days sooner than with the wild-type OSKM cocktail.

Validation Across Cell Types and Delivery Methods

To test clinical potential, the team tested mRNA delivery (instead of viral vectors) and mesenchymal stromal cells (MSCs) from three middle-aged human donors (over 50 years old). The findings included:

  • Rapid Expression: Within 7 days, over 30% of cells expressed pluripotency markers (SSEA4 and TRA-1-60).
  • High Activation: By day 12, over 85% of cells activated endogenous expression of OCT4, NANOG, SOX2, and TRA-1-60.
  • Stability: The resulting iPSCs could differentiate into all three primary germ layers and maintained healthy karyotypes and genomic stability.

Enhanced DNA Damage Repair and Rejuvenation

Beyond reprogramming efficiency, the researchers investigated the rejuvenation potential of the engineered variants by examining their ability to restore youthful characteristics to aged cells, specifically focusing on DNA damage (a hallmark of aging).

In a DNA-damage assay using $\gamma$-H2AX immunostaining to quantify double-strand breaks, cells treated with the RetroSOX/KLF cocktail showed significantly less $\gamma$-H2AX intensity than cells reprogrammed with standard OSKM or a fluorescent control. This indicates that the engineered variants reduce DNA damage more effectively than the original Yamanaka factors, suggesting a potential path toward improved cell rejuvenation therapies.

Sources