Gemma Scope 2 release – open tools for interpreting Gemma 3 language models

TL;DR

Gemma Scope 2 is an open‑source suite of interpretability tools for the entire Gemma 3 family (270 M–27 B parameters), enabling researchers to trace internal computations and debug safety‑critical behaviours such as jailbreaks, hallucinations, and sycophancy.

What Gemma Scope 2 Is

Gemma Scope 2 is a comprehensive, publicly released toolkit that lets AI safety researchers inspect the inner workings of Gemma 3 language models. It builds on the original Gemma Scope by adding full‑scale coverage, refined analysis components, and specialised tools for chat‑tuned variants.

  • Scope – Supports every Gemma 3 model size from 270 M to 27 B parameters.
  • Open‑source – All code, data, and pretrained components are released under an open licence, making it the largest interpretability release from an AI lab to date.
  • Data volume – The release required storing roughly 110 PB of intermediate data and training more than 1 trillion parameters for the auxiliary models.

Core Technical Advances

Sparse Autoencoders and Transcoders on Every Layer

Gemma Scope 2 trains Sparse Autoencoders (SAEs) and transcoders for each transformer layer of the Gemma 3 family. These components act as a microscope, mapping high‑dimensional activation patterns to human‑readable concepts.

New Decoder Architectures

  • Skip‑transcoders – Allow direct mapping from a layer’s activations to downstream layers, simplifying the reconstruction of multi‑step computations.
  • Cross‑layer transcoders – Enable attribution of behaviours that span several layers, making it easier to follow algorithms that are distributed across the network.

Matryoshka Training Technique

Gemma Scope 2 adopts the Matryoshka training technique (see arXiv:2503.17547) to improve concept discovery in SAEs. This method iteratively refines encoders, reducing redundancy and increasing the semantic purity of extracted features.

Chat‑Specific Behaviour Tools

Dedicated analysis pipelines target Gemma 3 models fine‑tuned for chat. Researchers can now isolate and visualise internal pathways responsible for:

  • Jailbreak attempts and refusal logic
  • Chain‑of‑thought reasoning fidelity
  • Sycophantic alignment with user prompts

Example Visualisations

The release includes visualisations such as the “Online scams and fraudulent emails” feature, where highlighted text fragments are coloured by activation strength, demonstrating how the model detects scam‑related concepts.

Feature visualisation

Why It Matters for AI Safety

  • Debugging emergent behaviours – By exposing the internal algorithmic steps, researchers can pinpoint why a model generates harmful outputs, facilitating targeted mitigations.
  • Auditing large models – Full‑scale coverage means safety analyses are no longer limited to small prototypes; behaviours that only appear in 27 B‑parameter models can now be examined.
  • Accelerating safety interventions – Open tools lower the barrier for the community to develop jailbreak detectors, hallucination reducers, and alignment checks, fostering collaborative progress.

Resources for Researchers

  • Download – The full package is hosted on Hugging Face: https://huggingface.co/google/gemma-scope-2
  • Technical report – Detailed methodology and evaluation are documented in the Gemma Scope 2 paper (PDF link in the original post).
  • Interactive demo – An online Neuronpedia demo lets users explore activations without installing the toolkit.
  • Colab tutorial – A ready‑to‑run notebook guides users through basic analyses and visualisations.

Future Directions

DeepMind expects the community to use Gemma Scope 2 to:

  1. Identify and mitigate jailbreak vectors in upcoming LLM releases.
  2. Study hallucination mechanisms by correlating internal concepts with erroneous outputs.
  3. Explore alignment gaps between a model’s stated reasoning and its latent computations.
  4. Inform the design of safer training regimes based on empirical insights from interpretability.

By providing unprecedented access to the “brain” of modern language models, Gemma Scope 2 aims to make safety‑critical research more reproducible, transparent, and impactful.

Related Links

Sources