Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning

Li, Qimai; Han, Zhichao; Wu, Xiao-Ming

doi:10.48550/arxiv.1801.07606

Cited by 70 publications

(74 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In other words, here in Section 3.1, we ignore the nonlinear activation and bias parameters. Such setup is consistent with many existing literature such as [30,33,34,60]. Proposition 3.1.…”

Section: Expressivity Analysis On Shadow-gcn: Graph Signal Processing...supporting

confidence: 90%

“…GCNs [22] suffer from "oversmoothing" [30] -Each GCN layer smooths the features of the direct (i.e., 1-hop) neighbors, and many GCN layers smooths the features of the full graph. Eventually, such repeated smoothing process propagates to any target node just the averaged feature of all V. "Oversmoothing" thus incurs significant information loss by wiping out all local information.…”

Section: Expressivity Analysis On Shadow-gcn: Graph Signal Processing...mentioning

confidence: 99%

“…• Expressivity challenge (i.e., oversmoothing [30,36,39,17]): iterative mixing of neighbor features collapses embedding vectors of different nodes into a fixed, low-dimensional subspace.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Decoupling the Depth and Scope of Graph Neural Networks

Zeng

Zhang

Xia

et al. 2022

Preprint

View full text Add to dashboard Cite

State-of-the-art Graph Neural Networks (GNNs) have limited scalability with respect to the graph and model sizes. On large graphs, increasing the model depth often means exponential expansion of the scope (i.e., receptive field). Beyond just a few layers, two fundamental challenges emerge: 1. degraded expressivity due to oversmoothing, and 2. expensive computation due to neighborhood explosion. We propose a design principle to decouple the depth and scope of GNNs -to generate representation of a target entity (i.e., a node or an edge), we first extract a localized subgraph as the bounded-size scope, and then apply a GNN of arbitrary depth on top of the subgraph. A properly extracted subgraph consists of a small number of critical neighbors, while excluding irrelevant ones. The GNN, no matter how deep it is, smooths the local neighborhood into informative representation rather than oversmoothing the global graph into "white noise". Theoretically, decoupling improves the GNN expressive power from the perspectives of graph signal processing (GCN), function approximation (GraphSAGE) and topological learning (GIN). Empirically, on seven graphs (with up to 110M nodes) and six backbone GNN architectures, our design achieves significant accuracy improvement with orders of magnitude reduction in computation and hardware cost.

show abstract

“…In other words, here in Section 3.1, we ignore the nonlinear activation and bias parameters. Such setup is consistent with many existing literature such as [30,33,34,60]. Proposition 3.1.…”

Section: Expressivity Analysis On Shadow-gcn: Graph Signal Processing...supporting

confidence: 90%

Section: Expressivity Analysis On Shadow-gcn: Graph Signal Processing...mentioning

confidence: 99%

See 1 more Smart Citation

Decoupling the Depth and Scope of Graph Neural Networks

Zeng

Zhang

Xia

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…First, we theoretically show that with standard degree normalization, low-degree neighbors surprisingly have more influence on a node's representation than higher-degree neighbors. Second, we discuss the results showing the dependency of GNNs on links within graph communities [10,14], which are ubiquitous in real-world graphs. Based on that, we argue that adversarial edges should link nodes with longer paths between them.…”

Section: Gnn Gnn Gnnmentioning

confidence: 99%

“…Communities are densely connected subgraphs, and are very common in empirical networks, e.g., social networks and co-author networks. The GNN update rule (Equation 1) is a special form of Laplacian smoothing [14]. Crucially, GNNs assume that nodes within the same community tend to share the same label and have similar features [10,14].…”

Section: Impact Of Degree and Distancementioning

confidence: 99%

Structack: Structure-based Adversarial Attacks on Graph Neural Networks

Hussain

Đuričić

Lex

et al. 2021

Proceedings of the 32st ACM Conference on Hypertext and Social Media

View full text Add to dashboard Cite

Recent work has shown that graph neural networks (GNNs) are vulnerable to adversarial attacks on graph data. Common attack approaches are typically informed, i.e. they have access to information about node attributes such as labels and feature vectors. In this work, we study adversarial attacks that are uninformed, where an attacker only has access to the graph structure, but no information about node attributes. Here the attacker aims to exploit structural knowledge and assumptions, which GNN models make about graph data. In particular, literature has shown that structural node centrality and similarity have a strong influence on learning with GNNs. Therefore, we study the impact of centrality and similarity on adversarial attacks on GNNs. We demonstrate that attackers can exploit this information to decrease the performance of GNNs by focusing on injecting links between nodes of low similarity and, surprisingly, low centrality. We show that structure-based uninformed attacks can approach the performance of informed attacks, while being computationally more efficient. With our paper, we present a new attack strategy on GNNs that we refer to as Structack. Structack can successfully manipulate the performance of GNNs with very limited information while operating under tight computational constraints. Our work contributes towards building more robust machine learning approaches on graphs. CCS CONCEPTS• Computing methodologies → Machine learning; Adversarial learning; Neural networks.

show abstract

Formula Graph Self‐Attention Network for Representation‐Domain Independent Materials Discovery

Ihalage

Hao

2022

Advanced Science

View full text Add to dashboard Cite

The success of machine learning (ML) in materials property prediction depends heavily on how the materials are represented for learning. Two dominant families of material descriptors exist, one that encodes crystal structure in the representation and the other that only uses stoichiometric information with the hope of discovering new materials. Graph neural networks (GNNs) in particular have excelled in predicting material properties within chemical accuracy. However, current GNNs are limited to only one of the above two avenues owing to the little overlap between respective material representations. Here, a new concept of formula graph which unifies stoichiometry-only and structure-based material descriptors is introduced. A self-attention integrated GNN that assimilates a formula graph is further developed and it is found that the proposed architecture produces material embeddings transferable between the two domains. The proposed model can outperform some previously reported structure-agnostic models and their structure-based counterparts while exhibiting better sample efficiency and faster convergence. Finally, the model is applied in a challenging exemplar to predict the complex dielectric function of materials and nominate new substances that potentially exhibit epsilon-near-zero phenomena.

show abstract

Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning

Cited by 70 publications

References 2 publications

Decoupling the Depth and Scope of Graph Neural Networks

Decoupling the Depth and Scope of Graph Neural Networks

Structack: Structure-based Adversarial Attacks on Graph Neural Networks

Formula Graph Self‐Attention Network for Representation‐Domain Independent Materials Discovery

Contact Info

Product

Resources

About