Survey of Image Based Graph Neural Networks

Nazir, Usman; Wang, He; Taj, Murtaza

doi:10.48550/arxiv.2106.06307

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, many scholars have combed and summarized GNNs from different perspectives (such as methods, applications, etc.). For details, please refer to the review [100][101][102][103][104][105][106][107][108][109][110]. Due to its high degree of freedom, good computability, and high reasoning efficiency, the spatial-based method has been widely concerned and developed.…”

Section: Output Layermentioning

confidence: 99%

Graph-based Knowledge Distillation: A survey and experimental evaluation

Liu¹,

Tongya²,

Zhang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Graph data, such as citation networks, social networks, and transportation networks, are prevalent in the real world. Graph neural networks (GNNs) have gained widespread attention for their robust expressiveness and exceptional performance in various graph analysis applications. However, the efficacy of GNNs is heavily reliant on sufficient data labels and complex network models, with the former being challenging to obtain and the latter requiring expensive computational resources. To address the labeled data scarcity and high complexity of GNNs, Knowledge Distillation (KD) has been introduced to enhance existing GNNs. This technique involves transferring the soft-label supervision of the large teacher model to the small student model while maintaining prediction performance. Transferring the KD technique to graph data and graph-based knowledge is a major challenge. This survey offers a comprehensive overview of Graph-based Knowledge Distillation methods, systematically categorizing and summarizing them while discussing their limitations and future directions. This paper first introduces the background of graph and KD. It then provides a comprehensive summary of three types of Graph-based Knowledge Distillation methods, namely Graph-based Knowledge Distillation for deep neural networks (DKD), Graph-based Knowledge Distillation for GNNs (GKD), and Self-Knowledge Distillation based Graph-based Knowledge Distillation (SKD). Each type of method is further divided into knowledge distillation methods based on the output layer, middle layer, and constructed graph. Subsequently, various graph-based knowledge distillation algorithms' ideas are analyzed and compared, concluding with the advantages and disadvantages of each algorithm supported by experimental results. In addition, the applications of graph-based knowledge distillation in computer vision, natural language processing, recommendation systems, and other fields are listed. Finally, the development of graph-based knowledge distillation is summarized and prospectively discussed. We have also released related resources at https://github.com/liujing1023/Graph-based-Knowledge-Distillation.

show abstract

Section: Output Layermentioning

confidence: 99%

Graph-based Knowledge Distillation: A survey and experimental evaluation

Liu¹,

Tongya²,

Zhang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Jiang et al [112] studied how GNNs inherently capture structural information stored in knowledge graphs to understand the strengths and weaknesses of existing GNN paradigms. Nazir et al [202] studied the GNNs applied in image classification tasks, where images can be converted into superpixels that form region adjacency graphs. Wu et al [281] systematically organized existing research of GNNs for NLP into a new paradigm consisting of graph construction, graph representation learning, and graph-based encoder-decoder, especially for NLP problems that can be best represented with graph structures.…”

Section: Autonomousmentioning

confidence: 99%

Graph Neural Networks in IoT: A Survey

Dong¹,

Tang²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

The Internet of Things (IoT) boom has revolutionized almost every corner of people's daily lives: healthcare, home, transportation, manufacturing, supply chain, and so on. With the recent development of sensor and communication technologies, IoT devices including smart wearables, cameras, smartwatches, and autonomous vehicles can accurately measure and perceive their surrounding environment. Continuous sensing generates massive amounts of data and presents challenges for machine learning. Deep learning models (e.g., convolution neural networks and recurrent neural networks) have been extensively employed in solving IoT tasks by learning patterns from multi-modal sensory data. Graph Neural Networks (GNNs), an emerging and fast-growing family of neural network models, can capture complex interactions within sensor topology and have been demonstrated to achieve state-of-the-art results in numerous IoT learning tasks. In this survey, we present a comprehensive review of recent advances in the application of GNNs to the IoT field, including a deep dive analysis of GNN design in various IoT sensing environments, an overarching list of public data and source code from the collected publications, and future research directions. To keep track of newly published works, we collect representative papers and their open-source implementations and create a Github repository at GNN4IoT. CCS Concepts: • Human-centered computing → Ubiquitous and mobile computing systems and tools; • Computing methodologies → Neural networks.

show abstract

Survey of Image Based Graph Neural Networks

Cited by 2 publications

References 0 publications

Graph-based Knowledge Distillation: A survey and experimental evaluation

Graph-based Knowledge Distillation: A survey and experimental evaluation

Graph Neural Networks in IoT: A Survey

Contact Info

Product

Resources

About