decoderesearch/SAELens

What it solves

SAELens provides a standardized way to train and analyze sparse autoencoders (SAEs) to better understand the internal workings of neural networks. This helps researchers create safer and more aligned AI systems by making the "black box" of model activations more interpretable.

How it works

The library allows users to extract activations from PyTorch-based models—including those from Hugging Face Transformers, NNsight, or TransformerLens—and pass them through an SAE's encoding and decoding methods. It provides tools for training new SAEs or loading pre-trained ones to decompose model activations into interpretable features.

Who it’s for

AI researchers focused on mechanistic interpretability and AI safety, specifically those looking to decompose neural network activations into sparse, understandable components.

Highlights

• own training pipeline for creating custom sparse autoencoders.
• Support for a wide range of PyTorch-based models and frameworks.
• Access to a library of pre-trained SAEs.
• Integration with visualization tools like SAE-Vis and evaluation benchmarks like SAEBench.