Richard Sutton on AI Creativity and Discovery

Generative AI is a Mimic, Not a Discoverer

Generative AI trained by supervised learning is fundamentally incapable of making novel discoveries because it is designed to mimic existing data. While these systems can produce output that is either "novel" (through stochastic randomness) or "good" (by adhering to high-quality training data), they cannot produce output that is simultaneously novel and good.

In most practical applications, novelty is actually undesirable. When users request a summary or a factual answer, novelty is categorized as a "hallucination." While stochastic processing allows AI to generate fiction or entertainment that feels new, this is a result of random trajectories rather than a creative process. Because supervised learning lacks a mechanism to evaluate the quality of its own novelty at runtime, it cannot perform true discovery.

The Three Pillars of Discovery

True creativity and discovery require a process that goes beyond pattern recognition and prediction. Richard Sutton defines "Discovery" as a three-step cycle:

  1. Variation: Generating a variety of trajectories or behaviors (which can be partly informed and partly "blind").
  2. Evaluation: Assessing which of the generated variations actually work or provide value.
  3. Selective Retention: Keeping the variations that performed the best.

This cycle is mirrored in evolution by natural selection, the scientific method, and human learning (known in psychology as instrumental learning or operant conditioning). In machine learning, this is the essence of reinforcement learning (RL) and combinatorial search.

Why Supervised Learning Fails at Discovery

Supervised learning and standard backpropagation lack the critical Evaluation step. While a model may generate a variety of outputs due to stochasticity, there is no internal mechanism to evaluate those outputs and retain the best ones during the runtime process.

Sutton notes that while backpropagation uses random initialization for variation, this happens only once at the start. To address this limitation and maintain plasticity, Sutton's research group introduced "continual backpropagation," where less-used neurons are periodically re-initialized to random weights, allowing variation to persist throughout the learning process.

Systems Capable of True Discovery

Discovery is possible when AI is augmented with an explicit goal or an external evaluation harness. Sutton cites several examples of systems that have achieved "novel and good" results:

  • AlphaGo and AlphaZero: Created original strategies (e.g., Move 37) by evaluating game states against a clear objective (winning).
  • GT-Sophy: Outperformed humans in simulated racecar driving.
  • AlphaFold, AlphaProof, and Claude-Code: Brought genuine advances in science, mathematics, and programming by combining generation with evaluation.
  • RL-Lyft: Optimized ride-hailing assignments through reward-based learning.

Community Perspectives and Counterpoints

Discussion among technical peers highlights several nuances regarding the boundary between mimicry and discovery:

The Role of Agentic Harnesses

Some argue that the most successful modern AI applications, such as coding assistants, are not pure LLMs but "agentic harnesses." These systems close the loop by using a compiler or terminal to provide verifiable rewards, effectively implementing the generate-test-refine loop Sutton describes.

Compositional Generalization

Another perspective suggests that deep learning performs "compositional generalization," where models recombine existing abstractions and styles in ways that never appeared in the training data. In this view, the resulting composition can be meaningfully novel even if the individual ingredients are derivative.

The Definition of Creativity

Critics of the "mimicry" label argue that almost all human innovation is derivative to some extent. They suggest that the difference between AI and human discovery is not the presence of a loop, but rather the sophistication of the "taste" or evaluation criteria used to rank results.

"The novelty flickers into existence but, if its value is unrecognized, it flickers away and is lost." — Richard Sutton

Sources