deel-ai/xplique

👋 Xplique is a Neural Networks Explainability Toolbox

What it solves

Xplique is a Python toolkit designed to address the "black box" nature of complex neural network models. It provides a comprehensive set of tools to help developers and researchers understand how their models make decisions, why they produce specific outputs, and what features they are prioritizing.

How it works

The library is organized into several specialized modules that apply different explainability techniques:

  • Attributions Methods: Implements techniques like Grad-CAM, Saliency, and Integrated Gradients to highlight which parts of the input (images, tabular data, or time series) most influenced the model's prediction.
  • Feature Visualization: Generates inputs that maximize the activation of specific neurons, channels, or layers to reveal what the network has "learned" to recognize.
  • Concepts: Extracts human-understandable concepts from the model (using methods like CAV and CRAFT) to test if these concepts are useful for specific classes.
  • Metrics: Provides tools to evaluate the faithfulness and quality of the explanations produced by the attribution methods.
  • Example-based: Explains model behavior by retrieving similar, contrastive (counter-factual), or prototype examples from a dataset.

Who it’s for

It is primarily for AI practitioners and researchers using TensorFlow (optimal compatibility with version 2.15 or earlier) or PyTorch (via a practical wrapper) who need to audit, debug, or explain the internal logic of their neural networks.

Highlights

  • Broad Task Support: Works across classification, regression, object detection, and semantic segmentation.
  • Diverse Data Modalities: Supports images, time series, and tabular data.
  • Framework Flexibility: Native support for TensorFlow with a wrapper for PyTorch integration.
  • Comprehensive Toolset: Combines attribution, visualization, concept extraction, and quantitative metrics in one library.

Related

  • Project
    jacobgil/pytorch-grad-cam

    A PyTorch library for explainable AI in computer vision that provides a wide range of pixel attribution methods to visualize and diagnose model predictions.

  • Project
    Trusted-AI/AIX360

    An open-source library providing a comprehensive set of algorithms and metrics to improve the interpretability and explainability of machine learning models and datasets across various data modalities.

  • Project
    MAIF/shapash

    A Python library that makes machine learning models interpretable for everyone by providing user-friendly visualizations, an interactive web app, and audit reports.

  • Project
    understandable-machine-intelligence-lab/Quantus

    Quantus is an open‑source Python library (PyTorch/TensorFlow) that provides 35+ quantitative metrics for evaluating neural‑network explanation methods across six categories (faithfulness, robustness, localisation, complexity, randomisation, axiomatic). It enables reproducible, batch‑efficient assessment of XAI techniques for image, time‑series, and tabular data, with tutorials, docs, and community support.

  • Project
    inseq-team/inseq

    A PyTorch-based toolkit for post-hoc interpretability of sequence generation models, providing various feature attribution methods to explain model outputs.