The Importance of Local Labels Distribution and Dominance for Node Classification in Graph Neural Networks

This paper studies the problem of semi-supervised learning on graphs, which aims to incorporate ubiquitous unlabeled knowledge (e.g., graph topology, node attributes) with few-available labeled knowledge (e.g., node class) to alleviate the scarcity issue of supervised information on node classification. While promising results are achieved, existing works for this problem usually suffer from the poor balance of generalization and fitting ability due to the heavy reliance on labels or task-agnostic unsupervised information. To address the challenge, we propose a dual-channel framework for semi-supervised learning on G raphs via K nowledge T ransfer between independent supervised and unsupervised embedding spaces, namely GKT. Specifically, we devise a dual-channel framework including a supervised model for learning the label probability of nodes and an unsupervised model for extracting information from massive unlabeled graph data. A knowledge transfer head is proposed to bridge the gap between the generalization and fitting capability of the two models. We use the unsupervised information to reconstruct batch-graphs to smooth the label probability distribution on the graphs to improve the generalization of prediction. We also adaptively adjust the reconstructed graphs by encouraging the label-related connections to solidify the fitting ability. Since the optimization of the supervised channel with knowledge transfer contains that of the unsupervised channel as a constraint and vice versa, we then propose a meta-learning-based method to solve the bi-level optimization problem, which avoids the negative transfer and further improves the model’s performance. Finally, extensive experiments validate the effectiveness of our proposed framework by comparing state-of-the-art algorithms.

show abstract

HGNN: A Hybrid Graph Neural Network Based on Transfer Learning for Linguistic Steganalysis

Varol Arısoy

2024

Afyon Kocatepe University Journal of Sciences and Engineering

View full text Add to dashboard Cite

Steganography, especially in the form of text generation based on secret messages, has become a current research topic. It is more difficult to identify the hidden message when it embedded directly into the text without using a cover text, and it also has a higher embedding capacity. Owing to the high rate of imperceptibility and resistance to steganalysis of this type steganography, it is essential that steganalysis methods, generate better performance. Although the complexity of deep learning models increases the accuracy rate, it also increases the inference time. In this study, a linguistic steganalysis was performed with a lower inference time and a higher accuracy rate. In the developed model, first, differences between non-stega and steganographic texts were modelled by a finetuned Bert using the custom dataset. The disparity information obtained by fine-tuned model was distilled into 3 separate networks, BertGCN, BertGAT and BertGIN, for faster and more accurate inference. Then, these 3 distilled networks were combined through Transfer Learning to form a new model. Experiments demonstrates that the proposed model surpass other methods in terms of the accuracy (a success of 0.9879 at 3.22 bpw on text encoded through SAAC Encoding) and the effectiveness of inference (1.09 second).

show abstract

The Importance of Local Labels Distribution and Dominance for Node Classification in Graph Neural Networks

Cited by 3 publications

References 13 publications

The impact of low-cost molecular geometry optimization in property prediction via graph neural network

The impact of low-cost molecular geometry optimization in property prediction via graph neural network

A Dual-channel Semi-supervised Learning Framework on Graphs via Knowledge Transfer and Meta-learning

HGNN: A Hybrid Graph Neural Network Based on Transfer Learning for Linguistic Steganalysis

Contact Info

Product

Resources

About