A Simple Spectral Failure Mode for Graph Convolutional Networks

Priebe, Carey E.; Shen, Cencheng; Huang, Ningyuan; Chen, Tianyi

doi:10.48550/arxiv.2010.13152

Cited by 2 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep GNNs are related to over-smoothing and have the primary goal to incorporate higher-layer neighborhood information through iterative propagation. For example, SGC [34] and S 2 GC [25] attempt to capture higher-layer neighborhood information by applying 𝐾 th power of the graph convolution in a single neural network layer. APPNP [13] replaces the power of the graph convolution with the Personalized PageRank [20] and GDC [14] further extends APPNP by generalizing Personalized PageRank to an arbitrary graph diffusion process.…”

Section: Over-smoothing Problem and Deep Gnnsmentioning

confidence: 99%

“…These two stages are packed together and termed as one layer of graph convolution. Deep GNNs iteratively perform multiple graph convolutions to obtain a larger receptive field and thus incorporate information of neighborhoods in higher layers [3,8,17,25].…”

Section: Introductionmentioning

confidence: 99%

“…Stepping further, several methods propose deep GNNs to incorporate higher-layer neighborhood information through iterative propagation [13,14,25,34]. For example, GCNII [3] applies an initial residual connection and identity mapping to enable GCN to express a 𝐾 th -order polynomial filter with arbitrary coefficients.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Tree Decomposed Graph Neural Network

Wang

Derr

2021

Proceedings of the 30th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) have achieved significant success in learning better representations by performing feature propagation and transformation iteratively to leverage neighborhood information. Nevertheless, iterative propagation restricts the information of higher-layer neighborhoods to be transported through and fused with the lower-layer neighborhoods', which unavoidably results in feature smoothing between neighborhoods in different layers and can thus compromise the performance, especially on heterophily networks. Furthermore, most deep GNNs only recognize the importance of higher-layer neighborhoods while yet to fully explore the importance of multi-hop dependency within the context of different layer neighborhoods in learning better representations. In this work, we first theoretically analyze the feature smoothing between neighborhoods in different layers and empirically demonstrate the variance of the homophily level across neighborhoods at different layers. Motivated by these analyses, we further propose a tree decomposition method to disentangle neighborhoods in different layers to alleviate feature smoothing among these layers. Moreover, we characterize the multi-hop dependency via graph diffusion within our tree decomposition formulation to construct Tree Decomposed Graph Neural Network (TDGNN), which can flexibly incorporate information from large receptive fields and aggregate this information utilizing the multi-hop dependency. Comprehensive experiments demonstrate the superior performance of TDGNN on both homophily and heterophily networks under a variety of node classification settings. Extensive parameter analysis highlights the ability of TDGNN to prevent over-smoothing and incorporate features from shallow layers with deeper multi-hop dependencies, which provides new insights towards deeper graph neural networks. The implementation of TDGNN is available at https://github.com/YuWVandy/TDGNN.

show abstract

Section: Over-smoothing Problem and Deep Gnnsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%