Hierarchical Representation Learning in Graph Neural Networks With Node Decimation Pooling

Abstract

In graph neural networks (GNNs), pooling operators compute local summaries of input graphs to capture their global properties, and they are fundamental for building deep GNNs that learn hierarchical representations. In this work, we propose the Node Decimation Pooling (NDP), a pooling operator for GNNs that generates coarser graphs while preserving the overall graph topology. During training, the GNN learns new node representations and fits them to a pyramid of coarsened graphs, which is computed offline in a pre-processing stage.NDP consists of three steps. First, a node decimation procedure selects the nodes belonging to one side of the partition identified by a spectral algorithm that approximates the MAXCUT solution. Afterwards, the selected nodes are connected with Kron reduction to form the coarsened graph. Finally, since the resulting graph is very dense, we apply a sparsification procedure that prunes the adjacency matrix of the coarsened graph to reduce the computational cost in the GNN. Notably, we show that it is possible to remove many edges without significantly altering the graph structure.Experimental results show that NDP is more efficient compared to state-of-the-art graph pooling operators while reaching, at the same time, competitive performance on a significant variety of graph classification tasks.

Deep Graph Mapper: Seeing Graphs Through the Neural Lens

“…In this work, we introduce Structural Deep Graph Mapper (SDGM) 1 —an adaptation of Mapper ( Singh et al, 2007 ), an algorithm from the field of Topological Data Analysis (TDA) ( Chazal and Michel, 2017 ), to graph domains. First, we prove that SDGM is a generalization of pooling methods based on soft cluster assignments, which include state-of-the-art algorithms like minCUT ( Bianchi et al, 2019 ) and DiffPool ( Ying et al, 2018 ). Building upon this topological perspective of graph pooling, we propose two pooling algorithms leveraging fully differentiable and fixed PageRank-based “lens” functions, respectively.…”

“…The Top- k approach is explored in conjunction with jumping-knowledge networks ( Cangea et al, 2018 ), attention ( Huang et al, 2019 ; Lee et al, 2019 ) and self-attention for cluster assignment ( Ranjan et al, 2019 ). Similarly to DiffPool, the method suggested by Bianchi et al (2019) uses several loss terms to enforce clusters with strongly connected nodes, similar sizes and orthogonal assignments. A different approach is also proposed by Ma et al (2019) , who leverage spectral clustering.…”

“…We compare the performance of DMP and MPR to two other pooling methods that we identify mathematical connections with: minCUT ( Bianchi et al, 2019 ) and DiffPool ( Ying et al, 2018 ). Additionally, we include Graph U-Net ( Gao and Ji, 2019 ) in our evaluation, as it has been shown to yield competitive results while performing pooling from the perspective of a learnable node ranking; we denote this approach by Top- k in the remainder of this section.…”

“…The Top- k architecture is evaluated using the code provided in the official repository, where separate configurations are defined for each of the benchmarks. The minCUT architecture is represented by the sequence of operations described by Bianchi et al (2019) : MP(32)-pooling-MP(32)-pooling-MP(32)-GlobalAvgPool, followed by a linear softmax classifier. The MP(32) block represents a message-passing operation performed by a graph convolutional layer with 32 hidden units: where is the symmetrically normalized adjacency matrix and are learnable weight matrices representing the message passing and skip-connection operations within the layer.…”

“…One may consider a fixed GNN architecture with a different pooling layer for each baseline; alternatively, the whole architecture can be optimized for each type of pooling layer. The first option, more akin to the typical procedure for evaluating pooling layers in CNNs on image domains, is used in papers like minCUT ( Bianchi et al, 2019 ). The second option is more particular to GNNs and it is employed, for instance, by DiffPool ( Ying et al, 2018 ).…”