shap/shap

What is SHAP?

SHAP (SHapley Additive exPlanations) is a Python library that explains the predictions of machine‑learning models using concepts from game theory. It computes Shapley values – a way of fairly assigning credit to each input feature for a model’s output – and presents them as additive contributions that sum to the difference between the model’s prediction and a baseline (the average prediction over a reference dataset).

Why it matters

• Model‑agnostic – works with any predictor (trees, linear models, deep nets, NLP pipelines, etc.).
• Consistent & locally accurate – the only additive feature‑attribution method that satisfies several desirable theoretical properties.
• Rich visualisations – waterfall, force, beeswarm, bar, scatter, decision‑plot, and text explanations help users see how features push predictions up or down.

How to get it

pip install shap          # from PyPI
# or
conda install -c conda-forge shap

GPU‑accelerated tree explanations are available with the optional cuda extra:

pip install "shap[cuda]"

(Requires a CUDA toolkit on the host.)

Core functionality (quick tour)

1. Tree models (XGBoost, LightGBM, CatBoost, scikit‑learn, Spark)

import shap, xgboost
X, y = shap.datasets.california()
model = xgboost.XGBRegressor().fit(X, y)
explainer = shap.Explainer(model)   # auto‑detects TreeExplainer
shap_vals = explainer(X)
shap.plots.waterfall(shap_vals[0])   # single‑instance explanation

Visuals: waterfall, force, beeswarm, bar.

2. Natural‑language models (Hugging Face transformers)

import shap, transformers
model = transformers.pipeline('sentiment-analysis')
explainer = shap.Explainer(model)
shap_vals = explainer(["What a great movie!", "...if you have no taste."])
shap.plots.text(shap_vals[0, :, "POSITIVE"])

Shows which words contributed to a positive or negative sentiment.

3. Deep learning (TensorFlow/Keras, PyTorch) – DeepExplainer

import shap, numpy as np
background = x_train[np.random.choice(len(x_train), 100, replace=False)]
explainer = shap.DeepExplainer(model, background)
shap_vals = explainer.shap_values(x_test[:5])
shap.image_plot(shap_vals, -x_test[:5])

Produces pixel‑level attributions for image models.

4. Expected gradients – GradientExplainer

Combines Integrated Gradients and SHAP ideas; works with any TF/Keras or PyTorch model and can explain intermediate layers.

5. Model‑agnostic – KernelExplainer

import shap, sklearn
svm = sklearn.svm.SVC(kernel='rbf', probability=True).fit(X_train, y_train)
explainer = shap.KernelExplainer(svm.predict_proba, X_train, link='logit')
shap_vals = explainer.shap_values(X_test, nsamples=100)
shap.force_plot(explainer.expected_value[0], shap_vals[0][0], X_test.iloc[0])

Works for any black‑box function but is slower.

6. Interaction values

For tree models you can also retrieve pairwise feature interactions:

shap_inter = shap.TreeExplainer(model).shap_interaction_values(X)

The diagonal holds main effects; off‑diagonals show how two features jointly influence predictions.

Documentation & learning resources

• Official docs: https://shap.readthedocs.io
• Example notebooks covering tree, deep, gradient, linear, and kernel explainers (e.g., NHANES survival model, MNIST digit classification, VGG16 intermediate‑layer analysis).
• Citations: The README links to the original Nature Machine Intelligence and Nature Biomedical Engineering papers that introduced Tree SHAP and the broader SHAP framework.

Who should use SHAP?

• Data scientists who need to interpret model predictions for stakeholders or regulatory compliance.
• Researchers building explainable AI pipelines for tabular, text, or image data.
• Anyone wanting visual, quantitative insight into feature importance beyond global metrics.

---

TL;DR: SHAP is a mature, widely‑used Python library that turns any ML model’s predictions into clear, mathematically‑grounded feature attributions, with a suite of fast algorithms for trees, deep nets, and generic models, plus many ready‑to‑use visualisations.