Reinforced Causal Explainer for Graph Neural Networks

Abstract

Explainability is crucial for probing graph neural networks (GNNs), answering questions like "Why the GNN model makes a certain prediction?". Feature attribution is a prevalent technique of highlighting the explanatory subgraph in the input graph, which plausibly leads the GNN model to make its prediction. Various attribution methods have been proposed to exploit gradient-like or attention scores as the attributions of edges, then select the salient edges with top attribution scores as the explanation. However, most of these works make an untenable assumption -the selected edges are linearly independent -thus leaving the dependencies among edges largely unexplored, especially their coalition effect. We demonstrate unambiguous drawbacks of this assumptionmaking the explanatory subgraph unfaithful and verbose. To address this challenge, we propose a reinforcement learning agent, Reinforced Causal Explainer (RC-Explainer). It frames the explanation task as a sequential decision process -an explanatory subgraph is successively constructed by adding a salient edge to connect the previously selected subgraph. Technically, its policy network predicts the action of edge addition, and gets a reward that quantifies the action's causal effect on the prediction. Such reward accounts for the dependency of the newly-added edge and the previously-added edges, thus reflecting whether they collaborate together and form a coalition to pursue better explanations. It is trained via policy gradient to optimize the reward stream of edge sequences. As such, RC-Explainer is able to generate faithful and concise explanations, and has a better generalization power to unseen graphs. When explaining different GNNs on three graph classification datasets, RC-Explainer achieves better or comparable performance to state-of-the-art approaches w.r.t. two quantitative metrics: predictive accuracy, contrastivity, and safely passes sanity checks and visual inspections. Codes and datasets are available at https://github.com/xiangwang1223/reinforced causal explainer.

PAGE: Prototype-Based Model-Level Explanations for Graph Neural Networks

“…Furthermore, PGM-Explainer [13] presented a probabilistic graphical model so as to provide an explanation by investigating predictions of GNNs when the GNN's input is perturbed. In RC-Explainer [15], a reinforcement learning agent was presented to construct an explanatory subgraph by adding a salient edge to connect the previously selected subgraph at each step, where a reward is obtained according to the causal effect for each edge addition. Most recently, CF-GNNExplainer [14] presented a counterfactual explanation in the form of minimal perturbation to the input graph such that the model prediction changes.…”

“…Thus, fostering explainability for GNN models has become of recent interest as it enables a thorough understanding of the model's behavior as well as trust and transparency [3], [4]. Recent attempts to explain GNN models mostly highlight a subgraph structure within a given input graph that contributed most towards the underlying GNN model's prediction [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15]. These so-called instance-level explanation methods for GNNs provide an in-depth analysis given a graph instance [16].…”

DAG Matters! GFlowNets Enhanced Explainer For Graph Neural Networks

“…As for the perturbation-based approaches, GNNExplainer Ying et al (2019) is the first specific design for explanation of GNNs, which formulates an optimization task to maximize the mutual information between the GNN predictions and the distribution of poten-tial subgraphs. Unfortunately, GNNExplainer and Causal Screening Wang et al (2020) may lack a global view of explanations and be stuck at local optima. Even though PGExplainer Luo et al (2020) andGraphMask Schlichtkrull et al (2020) could provide some global insights, they require a reparameterization trick and could not guarantee that the outputs of the subgraph are connected, which lacks explanations for the message passing scheme in GNNs.…”

“…Instance-level approaches explain models by identifying the most critical input features for their predictions. They have four sub-branches: Gradients/Features-based Zhou et al (2016); Baldassarre & Azizpour (2019); Pope et al (2019), Perturbation-based Ying et al (2019); Luo et al (2020); Schlichtkrull et al (2020); Wang et al (2020), Decompose-based Baldassarre & Azizpour (2019); Schnake et al (2020); Feng et al (2021) and Surrogate-based Vu & Thai (2020); Huang et al (2022); . Some works such as XGNN Yuan et al (2020) and RGExplainer Shan et al (2021) apply reinforcement learning (RL) to model-level and instance-level explanations.…”

MixupExplainer: Generalizing Explanations for Graph Neural Networks with Data Augmentation

“…Many attempts have been made to interpret GNN models and explain their predictions [24,31,33,42,50,53]. These methods can be grouped into two categories based on granularity: (1) instance-level explanation, which explains the prediction for each instance by identifying significant substructures [31,50,53], and (2) modellevel explanation, which seeks to understand the global decision rules captured by the GNN [2,24,33].…”