Causal Attention for Interpretable and Generalizable Graph Classification

Sui, Yongduo; Wang, Xiang; Wu, Jiancan; Lin, Min; He, Xiangnan; Chua, Tat-Seng

doi:10.1145/3534678.3539366

Cited by 73 publications

(30 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Generalization Algorithms: Empirical Risk Minimization (ERM), IRM (Arjovsky et al, 2019), GroupDRO (Sagawa et al, 2020), VREx (Krueger et al, 2021). • Graph Invariant Learning and Generalization: DIR (Wu et al, 2022b), CAL (Sui et al, 2022), GSAT (Miao et al, 2022), OOD-GNN (Li et al, 2022a), StableGNN (Fan et al, 2021). • Graph Data Augmentation: DropEdge (Rong et al, 2020), GREA , FLAG (Kong et al, 2022), M-Mixup , G-Mixup .…”

Section: Methodsmentioning

confidence: 99%

“…Scrutinizing Problem 1, we observe that the covariate shift is mainly caused by the scarcity of training environments. Existing efforts Sui et al, 2022;Wu et al, 2022b) make intervention or replacement of the environments to capture causal features. However, these environmental features still stem from the training distribution, which may result in a limited diversity of the environments.…”

Section: Two Principles For Graph Augmentationmentioning

confidence: 99%

“…• Invariant graph learning (Wu et al, 2022b;Sui et al, 2022) gradually becomes a prevalent paradigm for OOD generalization. The main idea is to capture the causal features by minimizing the empirical risks within different environments.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Adversarial Causal Augmentation for Graph Covariate Shift

Sui¹,

Wang²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Out-of-distribution (OOD) generalization on graphs is drawing widespread attention. However, existing efforts mainly focus on the OOD issue of correlation shift. While another type, covariate shift, remains largely unexplored but is the focus of this work. From a data generation view, causal features are stable substructures in data, which play key roles in OOD generalization. While their complementary parts, environments, are unstable features that often lead to various distribution shifts. Correlation shift establishes spurious statistical correlations between environments and labels. In contrast, covariate shift means that there exist unseen environmental features in test data. Existing strategies of graph invariant learning and data augmentation suffer from limited environments or unstable causal features, which greatly limits their generalization ability on covariate shift. In view of that, we propose a novel graph augmentation strategy: Adversarial Causal Augmentation (AdvCA), to alleviate the covariate shift. Specifically, it adversarially augments the data to explore diverse distributions of the environments. Meanwhile, it keeps the causal features invariant across diverse environments. It maintains the environmental diversity while ensuring the invariance of the causal features, thereby effectively alleviating the covariate shift. Extensive experimental results with in-depth analyses demonstrate that AdvCA can outperform 14 baselines on synthetic and real-world datasets with various covariate shifts.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Two Principles For Graph Augmentationmentioning

confidence: 99%

See 1 more Smart Citation

Adversarial Causal Augmentation for Graph Covariate Shift

Sui¹,

Wang²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…One class of techniques performs interventions on the nodes (Knyazev et al, 2019;. Meanwhile, other methods such as CAL (Sui et al, 2021) or CFLP (Zhao et al, 2021) augment the loss by adding additional tasks intended to separate causal and non-causal effects.…”

Section: Related Workmentioning

confidence: 99%

“…The key conceptual advance of this work is to scalably leverage active interventions on node neighborhoods (i.e., deletion of specific edges) to align graph attention training with the causal impact of these interventions on task performance. While some efforts have previously been made to infuse notions of causality into GNNs, these causal approaches have been largely limited to using causal effects from pre-trained models as features for a separate model Knyazev et al, 2019) or decoupling causal from non-causal effects (Sui et al, 2021). Causality is a frequently misused term, and CAR may not uncover rigorously causal mechanisms.…”

Section: Introductionmentioning

confidence: 99%

Causally-guided Regularization of Graph Attention Improves Generalizability

Wu¹,

Markovich²,

Berger³

et al. 2022

Preprint

View full text Add to dashboard Cite

Graph attention networks estimate the relational importance of node neighbors to aggregate relevant information over local neighborhoods for a prediction task. However, the inferred attentions are vulnerable to spurious correlations and connectivity in the training data, hampering the generalizability of the model. We introduce CAR, a general-purpose regularization framework for graph attention networks. Embodying a causal inference approach, CAR aligns the attention mechanism with the causal effects of active interventions on graph connectivity in a scalable manner. CAR is compatible with a variety of graph attention architectures, and we show that it systematically improves generalizability on various node classification tasks. Our ablation studies indicate that CAR hones in on the aspects of graph structure most pertinent to the prediction (e.g., homophily), and does so more effectively than alternative approaches. Finally, we also show that CAR enhances interpretability of attention weights by accentuating node-neighbor relations that point to causal hypotheses. For social media network-sized graphs, a CAR-guided graph rewiring approach could allow us to combine the scalability of graph convolutional methods with the higher performance of graph attention.

show abstract

A survey of out‐of‐distribution generalization for graph machine learning from a causal view

2024

AI Magazine

View full text Add to dashboard Cite

Graph machine learning (GML) has been successfully applied across a wide range of tasks. Nonetheless, GML faces significant challenges in generalizing over out‐of‐distribution (OOD) data, which raises concerns about its wider applicability. Recent advancements have underscored the crucial role of causality‐driven approaches in overcoming these generalization challenges. Distinct from traditional GML methods that primarily rely on statistical dependencies, causality‐focused strategies delve into the underlying causal mechanisms of data generation and model prediction, thus significantly improving the generalization of GML across different environments. This paper offers a thorough review of recent progress in causality‐involved GML generalization. We elucidate the fundamental concepts of employing causality to enhance graph model generalization and categorize the various approaches, providing detailed descriptions of their methodologies and the connections among them. Furthermore, we explore the incorporation of causality in other related important areas of trustworthy GML, such as explanation, fairness, and robustness. Concluding with a discussion on potential future research directions, this review seeks to articulate the continuing development and future potential of causality in enhancing the trustworthiness of GML.

show abstract

Causal Attention for Interpretable and Generalizable Graph Classification

Cited by 73 publications

References 11 publications

Adversarial Causal Augmentation for Graph Covariate Shift

Adversarial Causal Augmentation for Graph Covariate Shift

Causally-guided Regularization of Graph Attention Improves Generalizability

A survey of out‐of‐distribution generalization for graph machine learning from a causal view

Contact Info

Product

Resources

About