BAIR Introducing SPEX and ProxySPEX for Scalable LLM Interaction Discovery

BAIR has introduced SPEX (Spectral Explainer) and ProxySPEX, two algorithms designed to identify influential interactions within Large Language Models (LLMs) at a scale previously computationally infeasible. By leveraging signal processing and coding theory, these frameworks allow researchers to move beyond isolating single components to understanding the complex dependencies that drive model behavior across thousands of features, data points, and internal components.

The Challenge of Complexity at Scale

Interpretability research typically focuses on three lenses: feature attribution (input features), data attribution (influential training examples), and mechanistic interpretability (internal model components). However, model behavior is rarely the result of isolated components; it emerges from complex interactions.

As the number of components grows, the potential interactions grow exponentially, making exhaustive analysis impossible. SPEX and ProxySPEX address this by exploiting two structural properties of influential interactions:

  1. Sparsity: Only a relatively small number of interactions actually drive the model's output.
  2. Low-degreeness: Influential interactions typically involve only a small subset of features.

By reframing interaction discovery as a sparse recovery problem, SPEX uses strategically selected ablations to combine candidate interactions and then employs decoding algorithms to isolate the specific drivers of a decision.

ProxySPEX: Improving Efficiency via Hierarchy

ProxySPEX builds upon the SPEX framework by identifying a third structural property: hierarchy. In complex machine learning models, if a higher-order interaction is important, its lower-order subsets are likely to be important as well.

By exploiting this hierarchical structure, ProxySPEX achieves performance matching that of SPEX while requiring approximately 10x fewer ablations, significantly reducing the computational cost of the interpretability process.

Applications in Feature Attribution

While standard feature attribution identifies individual important features, SPEX captures the relationships between them. This is critical for understanding nuances such as double negatives in sentiment analysis or the synthesis of multiple documents in Retrieval-Augmented Generation (RAG) tasks.

Faithfulness and Scalability

In sentiment analysis tasks, SPEX maintains high faithfulness—the accuracy with which recovered attributions predict model output on unseen test ablations—even as context scales to thousands of features. While marginal approaches like LIME and Banzhaf can scale, they exhibit lower faithfulness because they fail to capture these complex interactions.

Case Study: The Trolley Problem

When analyzing a modified trolley problem where GPT-4o mini failed (answering correctly only 8% of the time), standard SHAP attribution identified the word "trolley" as the primary driver of the error. However, replacing "trolley" with synonyms had little effect. SPEX revealed a high-order synergy between the words "trolley" (two instances), "pulling", and "lever". Replacing these four specific words with synonyms reduced the model's failure rate to near zero.

Data Attribution and Training Set Influence

ProxySPEX can identify how training data points interact to influence a prediction. This allows researchers to distinguish between two types of interactions:

  • Synergistic Interactions: Semantically distinct classes that work together to define a decision boundary. For example, a prediction for an automobile might be driven by the synergy of a sports car's low-profile chassis, a truck's boxy shape, and a delivery vehicle's horizontal stripe.
  • Redundant Interactions: Visual duplicates that reinforce a specific concept, such as a cluster of dog images with similar silhouettes influencing a "horse" prediction.

This distinction enables the development of data selection techniques that remove redundancies while preserving necessary synergies.

Mechanistic Interpretability and Attention Head Attribution

ProxySPEX allows for the identification of interactions between internal model components, such as attention heads.

Architectural Interventions

On an MMLU dataset (highschool-us-history), a pruning strategy informed by ProxySPEX outperformed competing methods and actually improved model performance on the target task.

Interaction Structure Across Depth

Analysis of model depth reveals a shift in how components interact:

  • Early Layers: Operate in a predominantly linear regime where attention heads contribute independently.
  • Later Layers: Interactions between attention heads become more pronounced, primarily occurring among heads within the same layer.

Implementation and Availability

The code for SPEX and ProxySPEX is integrated into the SHAP-IQ repository and is available for public use at https://github.com/mmschlk/shapiq.

Sources