Graph Representation Learning

Yang, Cheng; Lin, Yankai; Liu, Zhiyuan; Sun, Maosong

doi:10.1007/978-981-99-1600-9_6

Cited by 3 publications

(2 citation statements)

References 82 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Future work could experiment with alternating GNN and Transformer layers 69 or other more complex and advanced architectures. 70,71 We might even explore constructing dynamic graphs for genes to incorporate temporal information. Lastly, to address the issues of data scarcity and imbalance met in downstream tasks, beyond using pre-training and rich biological features, semi-supervised learning [72][73][74] and long-tail learning 75,76 may offer more solutions.…”

Section: Discussionmentioning

confidence: 99%

Cell-Graph Compass: Modeling Single Cells with Graph Structure Foundation Model

Fang,

Hu,

Chang

et al. 2024

Preprint

View full text Add to dashboard Cite

Inspired by the advancements in pre-trained Large Language Models, there has been a surge of studies in the Life Sciences focusing on constructing foundation models with large scale single-cell RNA-seq data. These studies typically involve pre-training a transformer model on large-scale single-cell sequencing data, followed by fine-tuning for a variety of downstream tasks, achieving notable performance. However, these models all share a common short-coming: to utilize the transformer architecture, originally designed for textual data, they artificially impose a sequential structure on genes within cells, simplifying the complex interactions between genes. Furthermore, they focus solely on transcriptomic data, neglecting other relevant biological information. To address these issues, here we introduce Cell-Graph Compass (CGC), the first foundational model that leverages graph structures to model single cells and describes cells from multiple perspectives, including transcriptional profiles, gene text summaries, transcription factor regulatory networks, gene co-expression patterns, and gene positional relationships. By incorporating self-attention mechanisms, we pretrained the model on 50 million human single-cell sequencing data, resulting in a robust digital representation of cells. Extensive downstream experiments demonstrate that our approach can capture meaningful biological knowledge and achieve superior results in various problem scenarios, achieving the state-of-the-art (SOTA).

show abstract

Section: Discussionmentioning

confidence: 99%

Cell-Graph Compass: Modeling Single Cells with Graph Structure Foundation Model

Fang,

Hu,

Chang

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Large Language Models (LLMs) have demonstrated significant success across various domains [4,8,12,16,18,19], largely attributed to their extensive knowledge memorized during the pretraining phase and their exceptional ability to generalize during the fine-tuning process on diverse textual datasets [2,7,9,13,14]. This success has spurred interest in combining graph neural networks (GNNs) with LLMs to enhance their capabilities in understanding and modeling graphs [6,11,15,21,23], including implementing LLMs as encoders to process features within GNNs [3,5,10,22], and employing LLMs as aligners with GNNs to enhance performance [1,20,24].…”

Section: Introductionmentioning

confidence: 99%

Can we Soft Prompt LLMs for Graph Learning Tasks?

Liu,

He,

Tian

et al. 2024

Companion Proceedings of the ACM Web Conference 2024

View full text Add to dashboard Cite

Graph plays an important role in representing complex relationships in real-world applications such as social networks, biological data and citation networks. In recent years, Large Language Models (LLMs) have achieved tremendous success in various domains, which makes applying LLMs to graphs particularly appealing. However, directly applying LLMs to graph modalities presents unique challenges due to the discrepancy and mismatch between the graph and text modalities. Hence, to further investigate LLMs' potential for comprehending graph information, we introduce Graph-Prompter, a novel framework designed to align graph information with LLMs via soft prompts. Specifically, GraphPrompter consists of two main components: a graph neural network to encode complex graph information and an LLM that effectively processes textual information. Comprehensive experiments on various benchmark datasets under node classification and link prediction tasks demonstrate the effectiveness of our proposed method. The GraphPrompter framework unveils the substantial capabilities of LLMs as predictors in graph-related tasks, enabling researchers to utilize LLMs across a spectrum of real-world graph scenarios more effectively 1 .

show abstract

Differentially Private Client Selection and Resource Allocation in Federated Learning for Medical Applications Using Graph Neural Networks

Messinis,

Protonotarios,

Doulamis

2024

Sensors

View full text Add to dashboard Cite

Federated learning (FL) has emerged as a pivotal paradigm for training machine learning models across decentralized devices while maintaining data privacy. In the healthcare domain, FL enables collaborative training among diverse medical devices and institutions, enhancing model robustness and generalizability without compromising patient privacy. In this paper, we propose DPS-GAT, a novel approach integrating graph attention networks (GATs) with differentially private client selection and resource allocation strategies in FL. Our methodology addresses the challenges of data heterogeneity and limited communication resources inherent in medical applications. By employing graph neural networks (GNNs), we effectively capture the relational structures among clients, optimizing the selection process and ensuring efficient resource distribution. Differential privacy mechanisms are incorporated, to safeguard sensitive information throughout the training process. Our extensive experiments, based on the Regensburg pediatric appendicitis open dataset, demonstrated the superiority of our approach, in terms of model accuracy, privacy preservation, and resource efficiency, compared to traditional FL methods. The ability of DPS-GAT to maintain a high and stable number of client selections across various rounds and differential privacy budgets has significant practical implications, indicating that FL systems can achieve strong privacy guarantees without compromising client engagement and model performance. This balance is essential for real-world applications where both privacy and performance are paramount. This study suggests a promising direction for more secure and efficient FL medical applications, which could improve patient care through enhanced predictive models and collaborative data utilization.

show abstract

Graph Representation Learning

Cited by 3 publications

References 82 publications

Cell-Graph Compass: Modeling Single Cells with Graph Structure Foundation Model

Cell-Graph Compass: Modeling Single Cells with Graph Structure Foundation Model

Can we Soft Prompt LLMs for Graph Learning Tasks?

Differentially Private Client Selection and Resource Allocation in Federated Learning for Medical Applications Using Graph Neural Networks

Contact Info

Product

Resources

About