meta-pytorch/captum

What is Captum?

Captum 是一个 open-source library，为 PyTorch 增加了模型可解释性工具。它实现了一系列归因算法（Integrated Gradients、DeepLift、Gradient Shap、SmoothGrad、TCAV、TracIn 等），让你能够询问 "which input features, neurons, or training examples contributed to this prediction?"。该库直接与任何 PyTorch 模型配合使用，包括使用 torchvision, torchtext 和其他领域特定扩展构建的模型。

---

Who is it for?

• Model developers：想要通过观察预测驱动因素来调试或改进其网络的模型开发者。
• Researchers：正在构建新的可解释性方法并需要基准测试套件的研究人员。
• Production engineers：需要为终端用户生成解释（例如：为什么做出某项建议）的生产环境工程师。

---

Quick installation

# from PyPI (most common)
pip install captum

# or via conda
conda install -c pytorch captum   # or conda-forge channel

最新开发版本安装方式：

git clone https://github.com/pytorch/captum.git
cd captum
pip install -e .

（使用 -e .[dev] 或 -e .[tutorials] 可添加额外的开发或教程依赖项。）

---

Minimal example (the README’s toy model)

import torch, torch.nn as nn
from captum.attr import IntegratedGradients, GradientShap, DeepLift, NoiseTunnel

class ToyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.lin1 = nn.Linear(3, 3)
        self.relu = nn.ReLU()
        self.lin2 = nn.Linear(3, 2)
        # deterministic weights for the demo
        self.lin1.weight = nn.Parameter(torch.arange(-4., 5.).view(3,3))
        self.lin1.bias   = nn.Parameter(torch.zeros(1,3))
        self.lin2.weight = nn.Parameter(torch.arange(-3., 3.).view(2,3))
        self.lin2.bias   = nn.Parameter(torch.ones(1,2))
    def forward(self, x):
        return self.lin2(self.relu(self.lin1(x)))

model = ToyModel().eval()
input   = torch.rand(2, 3)
baseline = torch.zeros(2, 3)

# Integrated Gradients
ig = IntegratedGradients(model)
attr, delta = ig.attribute(input, baseline, target=0, return_convergence_delta=True)
print('IG attributions:', attr)
print('Delta:', delta)

# GradientShap (uses a baseline distribution)
gs = GradientShap(model)
baseline_dist = torch.randn(10, 3) * 0.001
attr, delta = gs.attribute(input, stdevs=0.09, n_samples=4,
                           baselines=baseline_dist, target=0,
                           return_convergence_delta=True)
print('GradShap attributions:', attr)

# Smoothing with NoiseTunnel (SmoothGrad)
nt = NoiseTunnel(IntegratedGradients(model))
attr, delta = nt.attribute(input, nt_type='smoothgrad', stdevs=0.02,
                           nt_samples=4, baselines=baseline, target=0,
                           return_convergence_delta=True)
print('SmoothGrad IG:', attr)

这段代码会打印出每个特征的归因分数（正值 → 支持预测，负值 → 反对预测）以及一个衡量积分近似精度的 convergence delta

---

Main capabilities (as listed in the README)

• Attribution methods: Integrated Gradients, DeepLift, Gradient Shap, SmoothGrad/VarGrad, TCAV, TracIn, etc.

• Neuron- and layer-level analysis: NeuronConductance, LayerConductance 让你能够看到哪些内部单元起作用。

• Compatibility: 该库直接与任何 PyTorch 模型配合使用，包括 torchvision, torchtext 和自定义架构。

• Bench-marking: 研究人员可以与内置的基准测试套件进行比较新算法。

---

Where to learn more

• Official docs: https://captum.ai/
• Tutorials (install with pip install -e .[tutorials])
• FAQ: docs/faq.md
• Talks & papers linked in the README (NeurIPS 2019, KDD 2020, etc.)

---

Bottom line: Captum 是任何需要 explain PyTorch 模型的人的必备库，无论是出于研究、调试或生产级用户面向的解释。