Rethinking Graph Neural Architecture Search from Message-passing

Cai, Shaofei; Li, Liang; Deng, Jincan; Zhang, Beichen; Zha, Zheng-Jun; Su, Li; Huang, Qingming

doi:10.1109/cvpr46437.2021.00659

Cited by 43 publications

(19 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The majority of these methods focus on design the aggregation layers in GNNs with different search algorithms. For example, GraphNAS [12], Auto-GNN [59], AutoGM [50], DSS [26] and [32] learn to design aggregation layers with diverse dimensions, such as attention function, attention head number, embedding size, etc; SANE [58], SNAG [57] and AutoGraph [24] provide the extra skip connections learning; GNAS [3] and Policy-GNN [22] learn to select the best message passing layers. Apart from design aggregation layers, RE-MPNN [19] learns adaptive global pooling functions additionally.…”

Section: Graph Neural Architecture Searchmentioning

confidence: 99%

“…Representative methods DARTS [27] and SNAS [45] use the Softmax and the Gumble-Softmax functions as the relaxation function, respectively. The differentiable search algorithms are used in SANE [58], DSS [26] and GNAS [3] to relax the aggregation dimensions. However, it is difficult to relax the pooling operations because different candidate pooling operations generate different coarse graphs consisting of diverse nodes and edges.…”

Section: Graph Neural Architecture Searchmentioning

confidence: 99%

See 1 more Smart Citation

Pooling Architecture Search for Graph Classification

Wei

Zhao²,

Yao

et al. 2021

Proceedings of the 30th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

Graph classification is an important problem with applications across many domains, like chemistry and bioinformatics, for which graph neural networks (GNNs) have been state-of-the-art (SOTA) methods. GNNs are designed to learn node-level representation based on neighborhood aggregation schemes, and to obtain graphlevel representation, pooling methods are applied after the aggregation operation in existing GNN models to generate coarse-grained graphs. However, due to highly diverse applications of graph classification, and the performance of existing pooling methods vary on different graphs. In other words, it is a challenging problem to design a universal pooling architecture to perform well in most cases, leading to a demand for data-specific pooling methods in real-world applications. To address this problem, we propose to use neural architecture search (NAS) to search for adaptive pooling architectures for graph classification. Firstly we designed a unified framework consisting of four modules: Aggregation, Pooling, Readout, and Merge, which can cover existing human-designed pooling methods for graph classification. Based on this framework, a novel search space is designed by incorporating popular operations in human-designed architectures. Then to enable efficient search, a coarsening strategy is proposed to continuously relax the search space, thus a differentiable search method can be adopted. Extensive experiments on six real-world datasets from three domains are conducted, and the results demonstrate the effectiveness and efficiency of the proposed framework 1 .

show abstract

Section: Graph Neural Architecture Searchmentioning

confidence: 99%

Section: Graph Neural Architecture Searchmentioning

confidence: 99%

Pooling Architecture Search for Graph Classification

Wei

Zhao²,

Yao

et al. 2021

Proceedings of the 30th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

show abstract

“…Considering the search efficiency, the differentiable algorithm is proposed to search architectures with gradient descent. It relaxes the discrete search space into continuous and then treats the architecture search problem as a bi-level optimization problem [3,28,29,59].…”

Section: Graph Neural Architecture Searchmentioning

confidence: 99%

“…We provide the performance comparisons of the GCNII, PNA and MixHop baselines used in our experiment and used in PyG 3 . As shown in Table 7, our baselines can achieve considerable performance on top of the unified framework with the same evaluation stage which will be introduced in the following.…”

Section: B Details Of Experiments B1 Baselinesmentioning

confidence: 99%

Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspective

Wei,

Zhao,

2021

Preprint

View full text Add to dashboard Cite

In recent years, Graph Neural Networks (GNNs) have shown superior performance on diverse real-world applications. To improve the model capacity, besides designing aggregation operations, GNN topology design is also very important. In general, there are two mainstream GNN topology design manners. The first one is to stack aggregation operations to obtain the higher-level features but easily got performance drop as the network goes deeper. Secondly, the multiple aggregation operations are utilized in each layer which provides adequate and independent feature extraction stage on local neighbors while are costly to obtain the higher-level information. To enjoy the benefits while alleviating the corresponding deficiencies of these two manners, we learn to design the topology of GNNs in a novel feature fusion perspective which is dubbed F 2 GNN. To be specific, we provide a feature fusion perspective in designing GNN topology and propose a novel framework to unify the existing topology designs with feature selection and fusion strategies. Then we develop a neural architecture search method on top of the unified framework which contains a set of selection and fusion operations in the search space and an improved differentiable search algorithm. The performance gains on eight real-world datasets demonstrate the effectiveness of F 2 GNN. We further conduct experiments to show that F 2 GNN can improve the model capacity while alleviating the deficiencies of existing GNN topology design manners, especially alleviating the over-smoothing problem, by utilizing different levels of features adaptively. 1 CCS CONCEPTS• Information systems → Data mining; • Computing methodologies → Neural networks.

show abstract

“…Gabriele et al [2] propose Principal Neighbourhood Aggregation (PNA) by integrating multiple aggregators (e.g., mean, max, min) together via degree-scalers. Cai et al [1] propose Graph Neural Architecture Search (GNAS) to learn the optimal depth of message passing with max and sum neighbor aggregations.…”

Section: Introductionmentioning

confidence: 99%

Generalizing Aggregation Functions in GNNs:High-Capacity GNNs via Nonlinear Neighborhood Aggregators

Wang¹,

Jiang²

2022

Preprint

View full text Add to dashboard Cite

Graph neural networks (GNNs) have achieved great success in many graph learning tasks. The main aspect powering existing GNNs is the multi-layer network architecture to learn the nonlinear graph representations for the specific learning tasks. The core operation in GNNs is message propagation in which each node updates its representation by aggregating its neighbors' representations. Existing GNNs mainly adopt either linear neighborhood aggregation (mean, sum) or max aggregator in their message propagation. (1) For linear aggregators, the whole nonlinearity and network's capacity of GNNs are generally limited due to deeper GNNs usually suffer from over-smoothing issue. (2) For max aggregator, it usually fails to be aware of the detailed information of node representations within neighborhood. To overcome these issues, we re-think the message propagation mechanism in GNNs and aim to develop the general nonlinear aggregators for neighborhood information aggregation in GNNs. One main aspect of our proposed nonlinear aggregators is that they provide the optimally balanced aggregators between max and mean/sum aggregations. Thus, our aggregators can inherit both (i) high nonlinearity that increases network's capacity and (ii) detail-sensitivity that preserves the detailed information of representations together in GNNs' message propagation. Promising experiments on several datasets show the effectiveness of the proposed nonlinear aggregators.

show abstract

Rethinking Graph Neural Architecture Search from Message-passing

Cited by 43 publications

References 29 publications

Pooling Architecture Search for Graph Classification

Pooling Architecture Search for Graph Classification

Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspective

Generalizing Aggregation Functions in GNNs:High-Capacity GNNs via Nonlinear Neighborhood Aggregators

Contact Info

Product

Resources

About