Qwen-Scope Interpretability Toolkit Release
Qwen has introduced Qwen-Scope, an interpretability toolkit designed to decode the internal mechanisms of the Qwen3 and Qwen3.5 series models. By utilizing Sparse Autoencoders (SAEs), the toolkit decomposes dense hidden representations into sparse, disentangled, and interpretable features, enabling developers to move beyond post-hoc analysis to actively optimize model performance.
Technical Implementation of Qwen-Scope
Qwen-Scope employs Sparse Autoencoders (SAEs) inserted into the hidden layers of LLMs to impose sparsity constraints. This process transforms the model's dense internal computations into human-understandable concepts and patterns.
To ensure stable training and broad feature coverage, the Qwen team sampled 0.5B tokens from the pretraining data of the corresponding models. The toolkit includes 14 sets of SAEs covering 7 different LLMs, including both dense and Mixture-of-Experts (MoE) architectures from the Qwen3 and Qwen3.5 series.
Supported Models and SAE Configurations
| Name | Backbone Type | SAE Width | Expansion Factor | L0 |
|---|---|---|---|---|
| SAE-Res-Qwen3-1.7B-Base-W32K-L0_50 | Base | 32K | 16 | 50 |
| SAE-Res-Qwen3-1.7B-Base-W32K-L0_100 | Base | 32K | 16 | 100 |
| SAE-Res-Qwen3-8B-Base-W64K-L0_50 | Base | 64K | 16 | 50 |
| SAE-Res-Qwen3-8B-Base-W64K-L0_100 | Base | 64K | 16 | 100 |
| SAE-Res-Qwen3.5-2B-Base-W32K-L0_50 | Base | 32K | 16 | 50 |
| SAE-Res-Qwen3.5-2B-Base-W32K-L0_100 | Base | 32K | 16 | 100 |
| SAE-Res-Qwen3.5-9B-Base-W64K-L0_50 | Base | 64K | 16 | 50 |
| SAE-Res-Qwen3.5-9B-Base-W64K-L0_100 | Base | 64K | 16 | 100 |
| SAE-Res-Qwen3.5-27B-W80K-L0_50 | Instruct | 80K | 16 | 50 |
| SAE-Res-Qwen3.5-27B-W80K-L0_100 | Instruct | 80K | 16 | 100 |
| SAE-Res-Qwen3-30B-A3B-Base-W32K-L0_50 | Base | 32K | 16 | 50 |
| SAE-Res-Qwen3-30B-A3B-Base-W128K-L0_100 | Base | 128K | 64 | 100 |
| SAE-Res-Qwen3.5-35B-A3B-Base-W32K-L0_50 | Base | 32K | 16 | 50 |
| SAE-Res-Qwen3.5-35B-A3B-Base-W128K-L0_100 | Base | 128K | 64 | 100 |
Core Applications and Capabilities
Qwen-Scope allows for the targeted control and analysis of model behavior across four primary dimensions: inference, data, training, and evaluation.
Controllable Inference
By manipulating the activation of specific features, developers can achieve targeted control over inference outcomes—such as modifying language, entities, or style—without the need for explicit natural language instructions.
Data Classification and Synthesis
Qwen-Scope functions as a data labeling and classification tool by analyzing model representations.
- Classification: Using a small amount of seed data, the toolkit can identify highly relevant features for sample classification (e.g., identifying toxic text) without additional training, reducing dependence on large bootstrapping datasets.
- Synthesis: The toolkit identifies features that are rarely or never activated in existing data. By directionally synthesizing supplementary samples based on these inactive features, Qwen reports an improvement in training data efficiency ratio by approximately 15 times for covering long-tail capabilities.
Targeted Fine-Tuning and Training
Interpretability features are used to guide the training process during supervised fine-tuning (SFT) and reinforcement learning (RL):
- SFT: By pinpointing anomalous activation patterns associated with issues like code-switching (e.g., unexpected Chinese words in English responses), a specific loss function can be designed to reduce these undesirable responses.
- RL: To address infrequent issues like endless repetitive generation, the toolkit can amplify the corresponding anomalous activation features to increase the likelihood of sampling these cases during the RL stage.
Evaluation Optimization
Qwen-Scope analyzes feature coverage across test sets to identify redundancies and gaps. By calculating feature activation patterns across different benchmark datasets, the team found that some commonly used datasets exhibit overlapping coverage, allowing users to select test samples with higher coverage and lower evaluation costs.