Attention-based graph neural networks: a survey

Sun, Chengcheng; Li, Chenhao; Lin, Xiang; Zheng, Tianji; Meng, Fanrong; Rui, Xiaobin; Wang, Zhixiao

doi:10.1007/s10462-023-10577-2

Cited by 23 publications

(2 citation statements)

References 112 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sun et al [21] present a thorough survey on attention-enhanced Graph Neural Networks (GNNs), addressing a gap in systematic exploration. Introducing a novel two-level taxonomy, it categorizes attention-based GNNs into graph recurrent attention networks, graph attention networks, and graph transformers.…”

Section: Related Workmentioning

confidence: 99%

Performance Evaluation of Graph Neural Network-Based RouteNet Model with Attention Mechanism

Dhamala,

Dawadi,

Manzoni

et al. 2024

Future Internet

View full text Add to dashboard Cite

Graph representation is recognized as an efficient method for modeling networks, precisely illustrating intricate, dynamic interactions within various entities of networks by representing entities as nodes and their relationships as edges. Leveraging the advantage of the network graph data along with deep learning technologies specialized for analyzing graph data, Graph Neural Networks (GNNs) have revolutionized the field of computer networking by effectively handling structured graph data and enabling precise predictions for various use cases such as performance modeling, routing optimization, and resource allocation. The RouteNet model, utilizing a GNN, has been effectively applied in determining Quality of Service (QoS) parameters for each source-to-destination pair in computer networks. However, a prevalent issue in the current GNN model is their struggle with generalization and capturing the complex relationships and patterns within network data. This research aims to enhance the predictive power of GNN-based models by enhancing the original RouteNet model by incorporating an attention layer into its architecture. A comparative analysis is conducted to evaluate the performance of the Modified RouteNet model against the Original RouteNet model. The effectiveness of the added attention layer has been examined to determine its impact on the overall model performance. The outcomes of this research contribute to advancing GNN-based network performance prediction, addressing the limitations of existing models, and providing reliable frameworks for predicting network delay.

show abstract

Section: Related Workmentioning

confidence: 99%

Performance Evaluation of Graph Neural Network-Based RouteNet Model with Attention Mechanism

Dhamala,

Dawadi,

Manzoni

et al. 2024

Future Internet

View full text Add to dashboard Cite

show abstract

“…This approach enables GNNs to identify detailed features and broader structural patterns across images, enhancing tumor identification accuracy. GNNs' ability to understand complex topological relationships in MRI data surpasses traditional neural networks, leading to more precise segmentation crucial for diagnosis and treatment planning [6][7][8][9]. Saueressig et al (2020Saueressig et al ( , 2021 showcased GNN's efficacy in segmenting brain tumors from 3D MRI scans by treating the data as graphs.…”

Section: Introductionmentioning

confidence: 99%

Advancing Brain Tumor Segmentation with Spectral–Spatial Graph Neural Networks

Mohammadi,

Allali

2024

Applied Sciences

View full text Add to dashboard Cite

In the field of brain tumor segmentation, accurately capturing the complexities of tumor sub-regions poses significant challenges. Traditional segmentation methods usually fail to accurately segment tumor subregions. This research introduces a novel solution employing Graph Neural Networks (GNNs), enriched with spectral and spatial insight. In the supervoxel creation phase, we explored methods like VCCS, SLIC, Watershed, Meanshift, and Felzenszwalb–Huttenlocher, evaluating their performance based on homogeneity, moment of inertia, and uniformity in shape and size. After creating supervoxels, we represented 3D MRI images as a graph structure. In this study, we combined Spatial and Spectral GNNs to capture both local and global information. Our Spectral GNN implementation employs the Laplacian matrix to efficiently map tumor tissue connectivity by capturing the graph’s global structure. Consequently, this enhances the model’s precision in classifying brain tumors into distinct types: necrosis, edema, and enhancing tumor. This model underwent extensive hyper-parameter tuning to ascertain the most effective configuration for optimal segmentation performance. Our Spectral–Spatial GNN model surpasses traditional segmentation methods in accuracy for both whole tumor and sub-regions, validated by metrics such as the dice coefficient and accuracy. For the necrotic core, the Spectral–Spatial GNN model showed a 10.6% improvement over the Spatial GNN and 8% over the Spectral GNN. Enhancing tumor gains were 9.5% and 6.4%, respectively. For edema, improvements were 12.8% over the Spatial GNN and 7.3% over the Spectral GNN, highlighting its segmentation accuracy for each tumor sub-region. This superiority underscores the model’s potential in improving brain tumor segmentation accuracy, precision, and computational efficiency.

show abstract

LGS‐PPIS: A Local–Global Structural Information Aggregation Framework for Predicting Protein–Protein Interaction Sites

Zhai,

Xu,

et al. 2024

Proteins

View full text Add to dashboard Cite

Exploring protein–protein interaction sites (PPIS) is of significance to elucidating the intrinsic mechanisms of diverse biological processes. On this basis, recent studies have applied deep learning‐based technologies to overcome the high cost of wet experiments for PPIS determination. However, the existing methods still suffer from two limitations that remain to be solved. Firstly, the process of feature aggregation in most methods only took into account node features, but ignored the complex edge features of the target residue to its neighbor residues, resulting in insufficient local feature extraction. Secondly, such feature aggregation was limited to aggregating spatially adjacent residues, and could not capture the “remote” residues that played a critical role in determining PPIS, which can be summed up as the lack of global feature at the residue level. To break the above limitations, a local–global structural information aggregation framework, LGS‐PPIS, was proposed in this study, including two modules of edge‐aware graph convolutional network (EA‐GCN) and self‐attention integrated with initial residual and identity mapping (SA‐RIM), which achieved the aggregation of local and global information for PPIS prediction. Evaluation results of LGS‐PPIS showed that the proposed method outperformed state‐of‐the‐art deep learning methods on three widely used PPIS prediction benchmarks. Besides, the results of ablation experiments demonstrated that the local features from spatially adjacent residues and global features from “remote” residues separately captured by EA‐GCN and SA‐RIM could benefit the model performance. Among them, the former was shown to have a more significant role in the PPIS prediction.

show abstract

Attention-based graph neural networks: a survey

Cited by 23 publications

References 112 publications

Performance Evaluation of Graph Neural Network-Based RouteNet Model with Attention Mechanism

Performance Evaluation of Graph Neural Network-Based RouteNet Model with Attention Mechanism

Advancing Brain Tumor Segmentation with Spectral–Spatial Graph Neural Networks

LGS‐PPIS: A Local–Global Structural Information Aggregation Framework for Predicting Protein–Protein Interaction Sites

Contact Info

Product

Resources

About