TransformerLensOrg/TransformerLens

What it solves

TransformerLens is designed for mechanistic interpretability—the process of reverse engineering the algorithms a trained language model has learned from its weights. It addresses the lack of open-source tooling for digging into the internal activations of generative models, making it easier for researchers to understand how models actually function internally.

How it works

The library provides a bridge to load over 15,000 open-source language models across 140+ architecture families (including GPT-2 style models and experimental support for Mamba/SSM architectures). It exposes the model's internal activations, allowing users to cache these activations or use hook functions to edit, remove, or replace them in real-time as the model runs.

Who it’s for

It is built for AI researchers and developers interested in mechanistic interpretability who want to explore the inner workings of LLMs without needing massive compute resources or industry-grade infrastructure.

Highlights

• Broad Model Support: Supports thousands of models across numerous architecture families via TransformerBridge.
• Internal Activation Access: Ability to cache and manipulate internal activations via hooks.
• Architecture Variety: Includes support for standard transformers and experimental support for State Space Models (SSMs) like Mamba-1 and Mamba-2.
• Research-Ready: Specifically designed to facilitate exploratory research and reverse engineering of model weights.