PAIR-code/what-if-tool

Source code/webpage/demos for the What-If Tool

What it solves

The What-If Tool (WIT) provides a visual interface for analyzing and understanding "black-box" machine learning models. It allows users to explore how models behave on a dataset without requiring any code, making it easier to debug model performance, investigate fairness, and understand the impact of specific features on predictions.

How it works

WIT integrates as a plugin for TensorBoard, Jupyter, or Colab notebooks. It connects to a trained ML model (via TensorFlow Estimators, TensorFlow Serving, AI Platform Prediction, or a custom Python prediction function) and a dataset of examples.

Users can interact with the data and model in several ways:

  • Visualization: Uses Facets Dive to organize and slice datasets by features or inference results.
  • Counterfactuals: Finds the most similar example in the dataset that resulted in a different classification to see what would have changed the outcome.
  • Manual Editing: Allows users to edit feature values of a specific data point in the browser and immediately re-run inference to see how the prediction changes.
  • Performance Analysis: Generates ROC curves and confusion matrices, which can be sliced by specific features (e.g., by gender) to analyze fairness and performance across different subsets of the data.
  • Attribution: Visualizes feature attribution values (e.g., from SHAP or Integrated Gradients) to show which features most influenced a prediction.

Who it’s for

  • ML Researchers and Developers: For debugging strange results and investigating model fairness.
  • Non-technical Stakeholders: To understand model performance on a dataset using their own data.
  • Lay Users: To learn about machine learning through interactive exploration.

Highlights

  • No-code interface: Explore models and data visually without writing code.
  • Counterfactual exploration: One-click comparison with the closest example that has a different classification.
  • Fairness tooling: Slicable ROC curves and confusion matrices to detect bias.
  • Interactive editing: Edit data points in real-time to see immediate changes in model output.
  • Model comparison: Ability to compare the results of two different models on the same input data.

Related

  • Project
    benchflow-ai/awesome-evals

    A curated, annotated list of 443+ verified resources for building and evaluating AI agents, with a runnable code playbook, deep reading notes, and a must-read starter set.

  • Project
    requie/AI-Red-Teaming-Guide

    A comprehensive guide to adversarial testing and security evaluation for AI systems, providing frameworks and methodologies to identify vulnerabilities in LLMs and autonomous agents.

  • Project
    DataDog/pup

    Pup is a Rust‑based CLI that wraps Datadog’s full API surface, delivering JSON/YAML output and OAuth2‑secured access so AI agents or scripts can query, create, and manage metrics, logs, monitors, security data, incidents, CI/CD, cost, and many other observability resources.

  • Project
    xinxuxin/keystone-bench

    Keystone Bench is a Python library and data set that evaluates medical chat assistants on how they adapt when a single decisive fact in a patient‑doctor conversation is removed, contradicted, or altered. It builds on OpenAI’s HealthBench, adds 7,419 edited “twins” across nine families, provides clinician‑written rubrics, and uses a separate LLM judge to label model replies as acceptable or forbidden. The benchmark offers layered splits, reproducible reference results, and a full audit pipeline, enabling researchers to measure evidence‑sensitivity and robustness of AI assistants in clinical settings.