Anthropic Research: Distributed Representations, Composition, and Superposition

Anthropic research indicates that distributed representations in neural networks can be divided into two distinct concepts: composition and superposition. While both allow a network to represent information, they operate on different principles and create a trade-off that puts them in fundamental tension regarding how functions are linearly computed and how the network generalizes.

The Role of Distributed Representations in Mechanistic Interpretability

Decomposing representations into independent components is essential for understanding neural networks and overcoming the "curse of dimensionality." By analyzing how models represent data, researchers can move from opaque weights to an independent component analysis that an analyst can actually understand.

Composition vs. Superposition: A Conceptual Framework

Anthropic distinguishes between "composition" and "superposition" as two different ways of representing information across a set of neurons. These two notions of distributed representations have different properties regarding generalization and the types of functions that can be linearly computed from them.

Composition

Composition involves representing a concept by combining different independent features. For example, if a network needs to represent shapes of different colors, it might use a separate set of neurons for "red" and "square own" and combine them to represent "red square."

Superposition

Superposition is the mechanism by which a network represents more features than it has dimensions (neurons), by treating features as directions in space rather than single neurons. This allows the network to compress more information into a smaller number of neurons than a traditional local code would allow.

Analyzing Representation Codes

To illustrate these differences, Anthropic references the work of Thorpe (1989), Thorpe's examples of "local," "semi-local," ""semi-distributed," and "high-distributed" codes—traditionally viewed as a spectrum from local to distributed—can be reimagined as varying across two different dimensions: superposition and composition.

By focusing on binary activations, the research simplifies the space of possibilities while remaining a rich enough environment to explore how these two mechanisms interact and a how they network represents features.