MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing

Abu-El-Haija, Sami; Perozzi, Bryan; Kapoor, Amol; Alipourfard, Nazanin; Lerman, Kristina; Harutyunyan, Hrayr; Steeg, Greg Ver; Galstyan, Aram

doi:10.48550/arxiv.1905.00067

Cited by 24 publications

(54 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To improve the GNN performance while increasing the model depth, various layer architectures have been proposed. AS-GCN [18], DeepGCN [28], JK-net [50], MixHop [1], Snowball [33], DAGNN [31] and GCNII [34] all include some variants of residue connection, either across multiple layers or within a single layer. In principle, such architectures can also benefit the feature propagation of a deep SHADOW-GNN, since their design does not rely on a specific neighborhood (e.g., L-hop).…”

Section: Related Workmentioning

confidence: 99%

“…To address the expressivity challenge, most remedies focus on neural architecture exploration: [44,12,49,29] propose more expressive aggregation functions when propagating neighbor features. [50,28,18,34,1,33,31] use residue-style design components to construct flexible and dynamic receptive fields. Among them, [50,28,18] use skip-connection across multiple GNN layers, and [34,1,33,31] encourage multi-hop message passing within each single layer.…”

Section: Introductionmentioning

confidence: 99%

“…[50,28,18,34,1,33,31] use residue-style design components to construct flexible and dynamic receptive fields. Among them, [50,28,18] use skip-connection across multiple GNN layers, and [34,1,33,31] encourage multi-hop message passing within each single layer. As for the scalability challenge, sampling methods have been explored to improve the training speed and efficiency.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Decoupling the Depth and Scope of Graph Neural Networks

Zeng

Zhang

Xia

et al. 2022

Preprint

View full text Add to dashboard Cite

State-of-the-art Graph Neural Networks (GNNs) have limited scalability with respect to the graph and model sizes. On large graphs, increasing the model depth often means exponential expansion of the scope (i.e., receptive field). Beyond just a few layers, two fundamental challenges emerge: 1. degraded expressivity due to oversmoothing, and 2. expensive computation due to neighborhood explosion. We propose a design principle to decouple the depth and scope of GNNs -to generate representation of a target entity (i.e., a node or an edge), we first extract a localized subgraph as the bounded-size scope, and then apply a GNN of arbitrary depth on top of the subgraph. A properly extracted subgraph consists of a small number of critical neighbors, while excluding irrelevant ones. The GNN, no matter how deep it is, smooths the local neighborhood into informative representation rather than oversmoothing the global graph into "white noise". Theoretically, decoupling improves the GNN expressive power from the perspectives of graph signal processing (GCN), function approximation (GraphSAGE) and topological learning (GIN). Empirically, on seven graphs (with up to 110M nodes) and six backbone GNN architectures, our design achieves significant accuracy improvement with orders of magnitude reduction in computation and hardware cost.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Decoupling the Depth and Scope of Graph Neural Networks

Zeng

Zhang

Xia

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The decrease of d may first reduce the irrelevant information from long-range context, and thus reducing the pose estimation error. When d takes an extreme value 1 16 , the useful information may also be substantially squeezed, leading to a performance drop. Impact of S and L. As shown in Tab.…”

Section: Ablation Studymentioning

confidence: 99%

“…To tackle this problem, geometric dependencies are incorporated into the network in [7,28,39,46], which significantly improve prediction accuracy. As articulated human body can be naturally modeled as a graph, with the recent development of graph neural networks (GNN) [11,13,1,41,47], various GNN-based methods [49,6,20,2,51] are proposed in the literature for 2D-to-3D pose estimation.…”

Section: Introductionmentioning

confidence: 99%

Learning Skeletal Graph Neural Networks for Hard 3D Pose Estimation

Zeng¹,

Sun²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Various deep learning techniques have been proposedto solve the single-view 2D-to-3D pose estimation problem. While the average prediction accuracy has been improved significantly over the years, the performance on hard poses with depth ambiguity, self-occlusion, and complex or rare poses is still far from satisfactory. In this work, we target these hard poses and present a novel skeletal GNN learning solution. To be specific, we propose a hop-aware hierarchical channel-squeezing fusion layer to effectively extract relevant information from neighboring nodes while suppressing undesired noises in GNN learning.In addition, we propose a temporal-aware dynamic graph construction procedure that is robust and effective for 3D pose estimation. Experimental results on the Human3.6M dataset show that our solution achieves 10.3% average prediction accuracy improvement and greatly improves on hard poses over state-of-the-art techniques. We further apply the proposed technique on the skeleton-based action recognition task and also achieve state-of-the-art performance. Our code is available at https://github. com/ailingzengzzz/Skeletal-GNN .

show abstract

Graph convolutional networks with higher‐order pooling for semisupervised node classification

Lei

Liu

Jiang

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

The information propagation mechanism in graph-structured networks such as social networks is the foundation of network security. The graph convolutional network (GCN) is a powerful approach for semisupervised node classification on graph-structure data. The vertex features which pass through the graph network are affected by the k-hop neighborhood vertices. However, current high-order GCN approaches merged the k-hop neighborhood using coarse pooling and complicated weight parameters. To reduce the computational complexity and preserve topological of the graph data, with weight sharing mechanism we propose a novel GCN based on a novel higher-order pooling layer for semisupervised classification. The proposed model and its variants are experimental studied on several large-scale citation network datasets using semisupervised learning. The experimental results show that the proposed model and its variants have lower computational complexity and achieve the state-of-the-art in the node classification accuracy.

show abstract

MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing

Cited by 24 publications

References 9 publications

Decoupling the Depth and Scope of Graph Neural Networks

Decoupling the Depth and Scope of Graph Neural Networks

Learning Skeletal Graph Neural Networks for Hard 3D Pose Estimation

Graph convolutional networks with higher‐order pooling for semisupervised node classification

Contact Info

Product

Resources

About