Anthropic Softmax Linear Units (SoLU) Research

Anthropic has introduced Softmax Linear Units (SoLU), a replacement for standard activation functions in MLP layers that increases the number of neurons responding to human-understandable concepts, phrases, or categories. This architectural change improves the model's interpretability without incurring a significant cost to overall machine learning performance.

The Softmax Linear Unit (SoLU) Architecture

SoLU is designed to replace the traditional activation function in the Multi-Layer Perceptron (MLP) layers of a transformer. By implementing this change, Anthropic researchers found that a significantly higher fraction of MLP neurons become "interpretable"—meaning they respond to specific, articulable properties of the input data.

Impact on Neuron Interpretability

Randomized and blinded experiments conducted by Anthropic demonstrate that SoLU models exhibit a more readily human-understandable internal state. Neurons in these models are more likely to correspond to clear categories or phrases, making it easier for researchers to understand how the model is processing information.

Implications for Transformer Information Processing

Using SoLU models, the researchers gained new insights into how information is processed within transformers. However, the study also revealed a technical trade-off regarding the feature representation of the model.

The Superposition Hypothesis and Feature Hiding

Anthropic's research indicates that SoLU may validate the superposition hypothesis. The evidence suggests that while SoLU makes certain features more interpretable by isolating them to specific neurons, it may simultaneously "hide" other features, making those remaining uninterpretable features even more deeply obscured.

Despite this trade-off, the researchers conclude that SoLU is a net win for interpretability research because it substantially increases the total fraction of neurons that humans can practically understand.