Graph Pointer Neural Networks

Abstract

Graph Neural Networks (GNNs) have shown advantages in various graph-based applications. Most existing GNNs assume strong homophily of graph structure and apply permutation-invariant local aggregation of neighbors to learn a representation for each node. However, they fail to generalize to heterophilic graphs, where most neighboring nodes have different labels or features, and the relevant nodes are distant. Few recent studies attempt to address this problem by combining multiple hops of hidden representations of central nodes (i.e., multi-hop-based approaches) or sorting the neighboring nodes based on attention scores (i.e., ranking-based approaches). As a result, these approaches have some apparent limitations. On the one hand, multi-hop-based approaches do not explicitly distinguish relevant nodes from a large number of multi-hop neighborhoods, leading to a severe over-smoothing problem. On the other hand, ranking-based models do not joint-optimize node ranking with end tasks and result in sub-optimal solutions. In this work, we present Graph Pointer Neural Networks (GPNN) to tackle the challenges mentioned above. We leverage a pointer network to select the most relevant nodes from a large amount of multi-hop neighborhoods, which constructs an ordered sequence according to the relationship with the central node. 1D convolution is then applied to extract high-level features from the node sequence. The pointer-network-based ranker in GPNN is joint-optimized with other parts in an end-to-end manner. Extensive experiments are conducted on six public node classification datasets with heterophilic graphs. The results show that GPNN significantly improves the classification performance of state-of-the-art methods. In addition, analyses also reveal the privilege of the proposed GPNN in filtering out irrelevant neighbors and reducing over-smoothing.

Disambiguated Node Classification with Graph Neural Networks

“…Graphs in real-world applications often exhibit heterophily, where a node tend to connect to nodes of different classes or features, reflecting diverse relationships or interactions. Heterophily has received lots of attention in recent years [59,68], and existing heterophilic GNNs are generally designed from two perspectives: (1) adding new nodes to the neighborhood to augment the propagated message; (2) flexible aggregation of neighborhood messages. Multi-hop neighbors are explored in [1,22,51,72] to augment the propagated messages, which tend to be more robust than using one-hop neighbors alone.…”

Graph Autoencoder with Preserving Node Attribute Similarity

“…Graph neural networks (GNNs) have recently become an important tool for graph representation learning. GNNs [ 11 , 28 , 29 , 30 , 31 ] mostly follow a recursive neighborhood scheme, using graph structural information to constrain each node to aggregate the attribute vectors of its neighborhood, and thus update its feature vector. After k aggregation iterations, a node representation is obtained, which fuses the structural and node attribute information within the node’s k -hop neighborhood.…”

Boosting Graph Structure Learning with Dummy Nodes

“…Besides, it also boosts learning to consider multihop neighbors (Zhu et al, 2020;Teru et al, 2020). On the other hand, dynamic high-order neighbor selection (Yang et al, 2021) and dynamic pointer links (Velickovic et al, 2020) illustrate the power of data-driven manipulations. Moreover, differentiable pooling yields consistent and significant performance improvement for end-to-end hierarchical graph representation learning (Ying et al, 2018;Zhang et al, 2019).…”