OpenAI Evolution through Large Models (ELM) Research

OpenAI has introduced Evolution through Large Models (ELM), a framework that leverages large language models (LLMs) to enhance the effectiveness of mutation operators in genetic programming (GP). By using LLMs to suggest likely code modifications, ELM can generate functional programs in domains where the model had no prior training data.

Improving Genetic Programming with LLMs

ELM replaces traditional, random mutation operators in genetic programming with mutations driven by a large language model. Because LLMs are trained on vast datasets containing sequential code changes and modifications, they can approximate the types of changes a human programmer would make. This allows the evolution of programs to move toward functional solutions more efficiently than random search.

Case Study: The Sodarace Domain

To demonstrate the breadth of ELM's capabilities, OpenAI researchers applied the framework to the Sodarace domain. The results include:

• Generation of Functional Programs: ELM, combined with the MAP-Elites algorithm, generated hundreds of thousands of functional Python programs.
• Novelty in Pre-training: These programs outputted working ambulating robots in a domain that the original LLM had never encountered during its pre-training phase.
• Bootstrapping New Models: The generated functional examples were used to bootstrap the training of a new conditional language model. This new model is capable of outputting the right "walker" (robot configuration) for a specific terrain given the context.

Implications for AI Research

The ability to bootstrap new models in domains with zero initial training data has significant implications for several fields of AI research:

• Open-endedness: ELM provides a mechanism for discovering new, functional artifacts in an environment without needing a labeled dataset.
• Deep Learning: The framework demonstrates how generative models can be used to create high-quality synthetic data to train subsequent models.
• Reinforcement Learning: The process of using evolutionary search to find successful behaviors and then distilling those behaviors into a neural network suggests new ways to approach complex reward functions and exploration.