Auto-GNN: Neural architecture search of graph neural networks

Abstract

Graph neural networks (GNNs) have been widely used in various graph analysis tasks. As the graph characteristics vary significantly in real-world systems, given a specific scenario, the architecture parameters need to be tuned carefully to identify a suitable GNN. Neural architecture search (NAS) has shown its potential in discovering the effective architectures for the learning tasks in image and language modeling. However, the existing NAS algorithms cannot be applied efficiently to GNN search problem because of two facts. First, the large-step exploration in the traditional controller fails to learn the sensitive performance variations with slight architecture modifications in GNNs. Second, the search space is composed of heterogeneous GNNs, which prevents the direct adoption of parameter sharing among them to accelerate the search progress. To tackle the challenges, we propose an automated graph neural networks (AGNN) framework, which aims to find the optimal GNN architecture efficiently. Specifically, a reinforced conservative controller is designed to explore the architecture space with small steps. To accelerate the validation, a novel constrained parameter sharing strategy is presented to regularize the weight transferring among GNNs. It avoids training from scratch and saves the computation time. Experimental results on the benchmark datasets demonstrate that the architecture identified by AGNN achieves the best performance and search efficiency, comparing with existing human-invented models and the traditional search methods.

AutoEdge-CCP: A novel approach for predicting cancer-associated circRNAs and drugs based on automated edge embedding

“…In this part, our model employs AutoGNN with Explicit Link Information [46] algorithm to construct edge feature engineering of the multi-source heterogeneous network. The AutoGNN model can automate the appropriate GNN architecture design for the given data [47] and introduce edge embedding in an explicit way. The edge feature engineering consists of the message passing phase and readout phase.…”

Multimodal Graph Neural Architecture Search under Distribution Shifts

“…Graph neural architecture search (GraphNAS) intends to automatically search for the most effective model architecture without human intervention (Qin et al 2022b;Zhang et al 2022bZhang et al , 2023. Existing GraphNAS methods can be categorized into reinforcement learning-based methods (Gao et al 2021b;Zhou et al 2022), evolutionary algorithms (Nunes and Pappa 2020;Shi et al 2022), and differentiable methods (Zhao et al 2020a,b). In recent years, scholars also explore how NAS performs under distribution shifts (Bai et al 2021;Qin et al 2022a).…”

RoSGAS : Adaptive Social Bot Detection with Reinforced Self-supervised GNN Architecture Search

“…It continuously generates descriptions of GNN architecture to find the optimal network architecture based on RL by maximizing the expected accuracy. Similarly to the architecture search in GraphNAS, Auto-GNN [63] additionally proposes a parameter sharing mechanism for sharing the parameters in homogeneous architecture for reducing the computation cost. However, these methods are not combined with the subgraph method and their optimization is tightly coupled with specific datasets.…”

“…To address these issues, the state-of-the-art works [20,27,52,62,63] adopt reinforcement learning (RL) to search the GNN architecture. However, such approaches sometimes lack generalizability; the effectiveness of determining the optimal GNN architecture is tightly bound to specific datasets and usually have huge search space, and hence low efficiency.…”