TransformerLensOrg/TransformerLens

What it solves

TransformerLens addresses the lack of open-source tooling for reverse-engineering the internal algorithms that generative language models learn during training. While many tools exist to train or use models, few provide the necessary infrastructure to easily inspect and manipulate their internal weights and activations.

How it works

The library provides a bridge to load over 15,000 open-source language models across 140+ architecture families. It exposes the model's internal activations, allowing researchers to cache these activations or use functions to edit, remove, or replace them in real-time as the model runs.

Who it’s for

It is designed for researchers and developers practicing mechanistic interpretability, enabling them to study how transformers and other architectures (like Mamba/SSMs) function internally without needing massive compute resources or industry-grade infrastructure.

Highlights

• Broad Model Support: Compatible with thousands of models across numerous architecture families, including GPT-2 style models and experimental support for Mamba-1 and Mamba-2.
• Activation Manipulation: Ability to cache internal activations and modify them during the model's forward pass.
• Research-Ready: Built specifically to facilitate the reverse-engineering of learned algorithms within trained models.
• Integration: Provides a TransformerBridge to maintain parity with HuggingFace weights while offering specialized interpretability tools.