Sampling Methods for Efficient Training of Graph Convolutional Networks: A Survey

Liu, Xin; Yan, Mingyu; Deng, Lei; Li, Guoqi; Ye, Xiaochun; Fan, Dongrui

doi:10.1109/jas.2021.1004311

Cited by 75 publications

(9 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To integrate information of transcriptomic profiles and spatial coordinates, PAST identifies k nearest neighbors ( k -NN) for each spot using spatial coordinates in a Euclidean space, and adopts GCNs to aggregate spatial patterns from each spot’s neighbors. Specifically, the self-attention mechanism 21 , which fits the relationship between words well in machine translation tasks 23 , is used to model local spatial patterns between neighboring spots, while the ripple walk sampler 22 , which enables efficient subgraph-based training for large and deep GCNs 24 , is used to achieve better scalability on large-scale ST data and preserve global spatial patterns simultaneously. PAST also restricts the distance of latent embeddings between neighbors through metric learning 25 , the insight of which is that spatially close spots are more likely to be positive pairs to show similar latent patterns (Methods).…”

Section: Resultsmentioning

confidence: 99%

PAST: latent feature extraction with a Prior-based self-Attention framework for Spatial Transcriptomics

Zhang

Chen

et al. 2022

Preprint

View full text Add to dashboard Cite

Rapid advances in spatial transcriptomics (ST) have revolutionized the interrogation of spatial heterogeneity and increased the demand for comprehensive methods to effectively characterize spatial domains. As a prerequisite for ST data analysis, spatial domain characterization is a crucial step for downstream analyses and biological implications. Here we propose PAST, a variational graph convolutional auto-encoder for ST, which effectively integrates prior information via a Bayesian neural network, captures spatial patterns via a self-attention mechanism, and enables scalable application via a ripple walk sampler strategy. Through comprehensive experiments on datasets generated by different technologies, we demonstrated that PAST could effectively characterize spatial domains and facilitate various downstream analyses, including ST visualization, spatial trajectory inference and pseudo-time analysis, by integrating spatial information and reference from various sources. Besides, we also show the advantages of PAST for accurate annotation of spatial domains in newly sequenced ST data and biological implications in the annotated domains.

show abstract

Section: Resultsmentioning

confidence: 99%

PAST: latent feature extraction with a Prior-based self-Attention framework for Spatial Transcriptomics

Zhang

Chen

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Due to its unique characteristics, GCN is able to capture features more comprehensively. Therefore, scholars have turned their attention to GCN and used effective training methods to improve their efficiency (Liu et al 2021). Bosselut et al ( 2019) with the COMET model, natural language can be used to generate diverse general knowledge.…”

Section: Event Relationship Extractionmentioning

confidence: 99%

Event Type and Relationship Extraction Based on Dependent Syntactic Semantic Augmented Graph Networks

Zuo,

Song,

Liu

et al. 2024

Preprint

View full text Add to dashboard Cite

In the domain of natural language processing, tasks such as trigger word extraction, event type recognition, and event relation extraction occupy pivotal positions. These tasks enable the discernment and extraction of salient information through profound analysis of textual content, thereby not only extricating key event data but also facilitating a deeper understanding of the semantic essence of the text. The process of extracting trigger words and event types often grapples with the complexities posed by polysemous words and intricate sentence structures, which can lead to deficiencies in the semantic representation of sentences. Addressing this challenge, this paper introduces a Dependency Syntactic Analysis model and proposes a novel framework Event Type Extraction Model base on Gravitational Network with Enhanced Dependency Semantics(GNEDS) that elucidates the intricate relationships and structures among words in a sentence. This approach significantly enriches the comprehension of contextual information, thereby enabling more precise identification of trigger words and their contextual affiliations, and consequently enhancing the text’s semantic representation.Furthermore, in the realm of event relation recognition, while traditional research has predominantly concentrated on intrasentential event relations, real world texts frequently exhibit event relations that span multiple sentences and entail complex contextual and implicit reasoning. This complexity often results in the sub-optimal performance of existing models in cross-sentence event relation extraction tasks. To overcome this limitation, this study introduces Graph Convolutional Neural Network (GCN) and the innovative concept of document nodes. Consequently, a Document Event Relationship Extraction based on Graph Convolutional Network with Enhanced Dependency Semantics(GCNEDS) is proposed that extends the scope to encompass a broader spectrum of global information, thereby enabling the amalgamation of textual information across various levels. This model is adept at capturing the long-distance dependencies between individual sentences within a document with greater accuracy, representing a significant advancement in the field of event type and relationship extraction based on dependent syntactic-semantic augmented graph networks.

show abstract

“…For a homogeneous graph, GNNs are widely used to learn node representation from the graph structure. The pioneer GCN (Kipf and Welling 2017) proposes a multi-layer network following a layer-wise propagation rule, where the l th layer learns an embedding vector h (Hamilton, Ying, and Leskovec 2017) improves the scalability for large graphs by introducing mini-batch training and neighbor sampling (Liu et al 2022a). GAT (Veličković et al 2018) introduces the attention mechanism to encourage the model to focus on the most important part of neighbors.…”

Section: Related Workmentioning

confidence: 99%

Simple and Efficient Heterogeneous Graph Neural Network

Yang

Yan

Pan

et al. 2023

AAAI

View full text Add to dashboard Cite

Heterogeneous graph neural networks (HGNNs) have the powerful capability to embed rich structural and semantic information of a heterogeneous graph into node representations. Existing HGNNs inherit many mechanisms from graph neural networks (GNNs) designed for homogeneous graphs, especially the attention mechanism and the multi-layer structure. These mechanisms bring excessive complexity, but seldom work studies whether they are really effective on heterogeneous graphs. In this paper, we conduct an in-depth and detailed study of these mechanisms and propose the Simple and Efficient Heterogeneous Graph Neural Network (SeHGNN). To easily capture structural information, SeHGNN pre-computes the neighbor aggregation using a light-weight mean aggregator, which reduces complexity by removing overused neighbor attention and avoiding repeated neighbor aggregation in every training epoch. To better utilize semantic information, SeHGNN adopts the single-layer structure with long metapaths to extend the receptive field, as well as a transformer-based semantic fusion module to fuse features from different metapaths. As a result, SeHGNN exhibits the characteristics of a simple network structure, high prediction accuracy, and fast training speed. Extensive experiments on five real-world heterogeneous graphs demonstrate the superiority of SeHGNN over the state-of-the-arts on both accuracy and training speed.

show abstract

Sampling Methods for Efficient Training of Graph Convolutional Networks: A Survey

Cited by 75 publications

References 67 publications

PAST: latent feature extraction with a Prior-based self-Attention framework for Spatial Transcriptomics

PAST: latent feature extraction with a Prior-based self-Attention framework for Spatial Transcriptomics

Event Type and Relationship Extraction Based on Dependent Syntactic Semantic Augmented Graph Networks

Simple and Efficient Heterogeneous Graph Neural Network

Contact Info

Product

Resources

About