awesome-xai

This paper provides an introduction to the social science research into explanations. The author provides 4 major findings: (1) explanations are constrastive, (2) explanations are selected, (3) probabilities probably don't matter, (4) explanations are social. These fit into the general theme that explanations are -contextual-

An important read for anyone using saliency maps. This paper proposes two experiments to determine whether saliency maps are useful: (1) model parameter randomization test compares maps from trained and untrained models, (2) data randomization test compares maps from models trained on the original dataset and models trained on the same dataset with randomized labels. They find that "some widely deployed saliency methods are independent of both the data the model was trained on, and the model parameters"

An in-depth description of XAI focused on technqiues for deep learning

An analysis of how different heatmap-based saliency methods perform based on experimentation with a generated dataset

Adaptive semantice inpute sampling for explanation

Accumulated local effects plot

Autoencoder Based Approach for Local Interpretability

High-Precision Model-Agnostic Explanations

Auditing black-box models

Bayesian local interpretable model-agnostic explanations

Break down plots for additive attributions

Class activation mapping

Confident interpretation of Bayesian decision tree ensembles

Centered ICE plot

Combined multiple models metalearner

Using sampling and queries to extract rules from trained neural networks

Contribution propogation

Extracting decision trees from trained neural networks

Deep label-specific feature learning for image annotation

Deep Taylor decomposition

Explanations of evidence in additive classifiers

Feature importance ranking measure

Meaninful perturbations model

Rule extraction using genetic algorithms

Explain random forest using decision tree

Exploring classifiers by randomization

Gaussian process decisions

Genetic program to evolve decision trees

Gradient-weighted Class Activation Mapping

Generalized gradient-based visual explanations

Making tree ensembles interpretable

Individual conditional expectation plots

Integrated gradients

Interpreting tree ensembles with inTrees

Iterative orthoganol feature projection

Information plane visualization

Kullback-Leibler Projections based LIME

Extracting decision trees from trained neural networks

Rationalizing neural predictions with generator and encoder

Local Interpretable Model-Agnostic Explanations

Leave-one covariate out

Local rule-based explanations

Accurate intelligibile models with pairwise interactions

Layer-wise relevance propogation

Model class reliance

Model explanation system

Feature importance measure for non-linear algorithms

Neural interpretation diagram

Optimized LIME

Partition aware local model

Prediction Difference Analysis: Visualize deep neural network decisions

Partial dependence plots

Positional oligomer importance matrices for understanding SVM signal detectors

Transfer information from deep network to simpler model

Interactive partial dependence diagnostics

Quantitative input influence

Extracting symbolic rules from trained neural network ensembles

Reverse time attention model

Randomized input sampling for explanation

Reverse engineering neural networks for rule extraction

A unified approach to interpretting model predictions

Similarity, difference, and uniqueness input perturbation

Visualizing CNN classes

Programs as black-box explanations

Interpreting models via Single Tree Approximation

Explanation of individual classifications using game theory

Rule extraction from support vector machines

Testing with concept activation vectors

Interpretable predictions of tree-ensembles via actionable feature tweaking

Making sense of a forest of trees

Consistent feature attribute for tree ensembles

Feature-space partitioning

Extracting tree-structured representations of trained networks

Tree space prototypes

Visual back-propagation

Variable effect characteristic curve

Variable interaction network

Adapted etraction of comprehensible decision tree in ANNs

Show, attend, tell attention model

Like a decision tree with no branches

The tree provides an interpretation

Method that predicts based on learned vector graphs of features

The prototypical clustering method

Easily plottable and understandable regression

Easily plottable and understandable classification

Good classification, poor estimation using conditional probabilities

Sparse linear model as decision rules including feature interactions

Authors perform a series of NLP experiments which argue attention does not provide meaningful explanations. They also demosntrate that different attentions can generate similar model outputs

This is a rebutal to the above paper. Authors argue that multiple explanations can be valid and that the and that attention can produce valid explanation, if not -the- valid explanation

Authors argue that addditive explanations (e.g. LIME, SHAP, Break Down) fail to take feature ineractions into account and are thus unreliable

Authors demonstrate why permuting features is misleading, especially where there is strong feature dependence. They offer several previously described alternatives

Authors present a number of issues with explainable ML and challenges to interpretable ML: (1) constructing optimal logical models, (2) constructing optimal sparse scoring systems, (3) defining interpretability and creating methods for specific methods. They also offer an argument for why interpretable models might exist in many different domains

Authors demonstrate how saliency methods vary attribution when adding a constant shift to the input data. They argue that methods should fulfill , that a saliency method mirror the sensistivity of the model with respect to transformations of the input

EthicalML/xai

A toolkit for XAI which is focused exclusively on tabular data. It implements a variety of data and model evaluation techniques

MAIF/shapash

SHAP and LIME-based front-end explainer

PAIR-code/what-if-tool

A tool for Tensorboard or Notebooks which allows investigating model performance and fairness

slundberg/shap

A Python module for using Shapley Additive Explanations

A debate on whether interpretability is necessary for ML with Rich Caruana and Patrice Simard for and Kilian Weinberger and Yann LeCun against

A UK-based research center that performs research into ethical AI/ML, which frequently involves XAI

One of the preeminent researchers in XAI

The man behind Explainable Boosting Machines