Second-Order Pooling for Graph Neural Networks

Abstract

Graph neural networks have achieved great success in learning node representations for graph tasks such as node classification and link prediction. Graph representation learning requires graph pooling to obtain graph representations from node representations. It is challenging to develop graph pooling methods due to the variable sizes and isomorphic structures of graphs. In this work, we propose to use second-order pooling as graph pooling, which naturally solves the above challenges. In addition, compared to existing graph pooling methods, second-order pooling is able to use information from all nodes and collect second-order statistics, making it more powerful. We show that direct use of second-order pooling with graph neural networks leads to practical problems. To overcome these problems, we propose two novel global graph pooling methods based on second-order pooling; namely, bilinear mapping and attentional second-order pooling. In addition, we extend attentional second-order pooling to hierarchical graph pooling for more flexible use in GNNs. We perform thorough experiments on graph classification tasks to demonstrate the effectiveness and superiority of our proposed methods. Experimental results show that our methods improve the performance significantly and consistently.

Distribution Knowledge Embedding for Graph Pooling

“…TU datasets. We follow the same settings in previous studies [9], [12] to perform 10-fold cross-validation for performance evaluation. To obtain graph node features, we adopt the GIN [2] and set the number of GIN layer as 5 in our experiments.…”

“…The most familiar averaging and summation operations in GNNs [2], [10], [11] belong to the flat graph pooling but only collect the first-order statistic information of the node representations and ignore the important higherorder statistics. Therefore, Wang et al [12] proposed a secondorder pooling framework as graph pooling and demonstrated the effectiveness and superiority of second-order statistics for graph neural networks. However, all these methods ignore non-Euclidean geometry characteristic of graphs, which will cause the massive information missing and downgrade the performance of graph representations.…”

“…Existing graph pooling methods can be broadly categorized into two schools, hierarchical graph pooling [6]- [9] and flat graph pooling [2], [10]- [12]. Hierarchical graph pooling iteratively operates on coarser and coarser representations of a graph, which involves large network modifications and can hardly be applied to off-the-shelf GNNs.…”

Learnable Structural Semantic Readout for Graph Classification

“…that outputs a summarized representation, in order to facilitate to combine it with our SSRead layer. We only consider the global aggregation functions officially implemented in PyTorch Geometric, but any other global aggregations, such as the second-order pooling [32], also can be combined with our SSRead layer to output the position-level representation.…”

An attention-driven hierarchical multi-scale representation for visual recognition

“…GCN in conjunction with attention mechanism has proven its effectiveness in modeling long-range dependency. Focusing on the node's most relevant hidden representations in a graph, a multi-head-attention [51] is exploited to stabilize the learning task, leveraging the second-order hierarchical pooling [57]. In [34], a self-attention is introduced for hierarchical pooling to learn structural information using the node features and graph topology.…”