DOTIN: Dropping Task-Irrelevant Nodes for GNNs

Abstract

Scalability is an important consideration for deep graph neural networks. Inspired by the conventional pooling layers in CNNs, many recent graph learning approaches have introduced the pooling strategy to reduce the size of graphs for learning, such that the scalability and efficiency can be improved. However, these pooling-based methods are mainly tailored to a single graph-level task and pay more attention to local information, limiting their performance in multi-task settings which often require task-specific global information. In this paper, departure from these poolingbased efforts, we design a new approach called DOTIN (Dropping Task-Irrelevant Nodes) to reduce the size of graphs. Specifically, by introducing K learnable virtual nodes to represent the graph embeddings targeted to K different graph-level tasks, respectively, up to 90% raw nodes with low attentiveness with an attention model -a transformer in this paper, can be adaptively dropped without notable performance decreasing. Achieving almost the same accuracy, our method speeds up GAT about 50% on graph-level tasks including graph classification and graph edit distance (GED) with about 60% less memory, on D&D dataset. Code will be made public available in https://github.com/Sherrylone/DOTIN.

Interpreting Unfairness in Graph Neural Networks via Training Node Attribution

“…Traditional bias metrics such as ∆ SP for statistical parity and ∆ EO for equal opportunity are computed on the predicted class labels. However, a single training node can hardly twist these predicted labels (Zhang et al 2022a;Sun et al 2020).…”

“…However, these metrics are not applicable in our task. The reason is that most of them are computed based on the predicted labels, while a single training node can barely twist these predicted labels on test data (Zhang et al 2022a;Sun et al 2020). Consequently, the influence of a single training node on the model bias would be hard to capture.…”