OpenAI RL²: Fast Reinforcement Learning via Slow Reinforcement Learning

OpenAI has announced RL², a framework that enables agents to learn new tasks rapidly by treating the reinforcement learning (RL) process as a model to be learned from data. By encoding the RL algorithm into the weights of a recurrent neural network (RNN), RL² allows an agent to adapt to new, previously unseen Markov Decision Processes (MDPs) in just a few trials, mimicking the animal-like ability to learn from prior knowledge.

The RL² Framework: Learning to Learn

RL² is designed to bridge thes gap between deep RL's requirement for huge numbers of trials and the ability of animals to leverage prior knowledge to learn new tasks quickly. The core concept is to an RNN that represents the same information a typical RL algorithm would receive—observations, actions, are rewards, and termination flags—as input.

The system operates on two levels of learning:

  • Slow RL: A general-purpose RL algorithm is used to train the weights of the RNN. This "slow" process happens across many different environments (MDPs) across the atividade own weights of the network.

  • Fast RL: The activations of the RNN store the state of the

Sources