Heterogeneous deep graph convolutional network with citation relational BERT for COVID-19 inline citation recommendation

Dai, Tao; Zhao, Jie; Li, Dehong; Shen, Tian; Zhao, Xiangmo; Pan, Shirui

doi:10.1016/j.eswa.2022.118841

Cited by 9 publications

(3 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Jeong et al [ 19 ] combined graph convolution networks with BERT representation of textual data to generate context-aware paper recommendations. Dai et al [ 20 ] introduced a two-stage COVID-19 paper citation recommender by enhancing BERT representation learning in the first stage, and learning effective dense vector of nodes among bibliographic graph through heterogenous deep graph convolutional networks. Hassen et al [ 21 ] compared several popular encoder models including USE, BERT, InferSent, ELMo and SciBERT and found out that solely semantic information from these models did not outperform BM25 for paper recommendations.…”

Section: Related Workmentioning

confidence: 99%

A novel NIH research grant recommender using BERT

Jie

Patra

et al. 2023

PLoS ONE

View full text Add to dashboard Cite

Research grants are important for researchers to sustain a good position in academia. There are many grant opportunities available from different funding agencies. However, finding relevant grant announcements is challenging and time-consuming for researchers. To resolve the problem, we proposed a grant announcements recommendation system for the National Institute of Health (NIH) grants using researchers’ publications. We formulated the recommendation as a classification problem and proposed a recommender using state-of-the-art deep learning techniques: i.e. Bidirectional Encoder Representations from Transformers (BERT), to capture intrinsic, non-linear relationship between researchers’ publications and grants announcements. Internal and external evaluations were conducted to assess the system’s usefulness. During internal evaluations, the grant citations were used to establish grant-publication ground truth, and results were evaluated against Recall@k, Precision@k, Mean reciprocal rank (MRR) and Area under the Receiver Operating Characteristic curve (ROC-AUC). During external evaluations, researchers’ publications were clustered using Dirichlet Process Mixture Model (DPMM), recommended grants by our model were then aggregated per cluster through Recency Weight, and finally researchers were invited to provide ratings to recommendations to calculate Precision@k. For comparison, baseline recommenders using Okapi Best Matching (BM25), Term-Frequency Inverse Document Frequency (TF-IDF), doc2vec, and Naïve Bayes (NB) were also developed. Both internal and external evaluations (all metrics) revealed favorable performances of our proposed BERT-based recommender.

show abstract

Section: Related Workmentioning

confidence: 99%

A novel NIH research grant recommender using BERT

Jie

Patra

et al. 2023

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…Designing link prediction algorithms that can efficiently handle largescale knowledge graphs is crucial for their practical applicability. Many existing link prediction methods involve complex models with numerous parameters, especially graphbased neural networks, such as GCNs [16] and R-GCNs [17]. Managing the number of parameters is essential to avoid overfitting and ensure the efficient training of models on large-scale knowledge graphs.…”

Section: Introductionmentioning

confidence: 99%

Convolutional Models with Multi-Feature Fusion for Effective Link Prediction in Knowledge Graph Embedding

Guo,

Liao,

et al. 2023

Entropy

View full text Add to dashboard Cite

Link prediction remains paramount in knowledge graph embedding (KGE), aiming to discern obscured or non-manifest relationships within a given knowledge graph (KG). Despite the critical nature of this endeavor, contemporary methodologies grapple with notable constraints, predominantly in terms of computational overhead and the intricacy of encapsulating multifaceted relationships. This paper introduces a sophisticated approach that amalgamates convolutional operators with pertinent graph structural information. By meticulously integrating information pertinent to entities and their immediate relational neighbors, we enhance the performance of the convolutional model, culminating in an averaged embedding ensuing from the convolution across entities and their proximal nodes. Significantly, our methodology presents a distinctive avenue, facilitating the inclusion of edge-specific data into the convolutional model’s input, thus endowing users with the latitude to calibrate the model’s architecture and parameters congruent with their specific dataset. Empirical evaluations underscore the ascendancy of our proposition over extant convolution-based link prediction benchmarks, particularly evident across the FB15k, WN18, and YAGO3-10 datasets. The primary objective of this research lies in forging KGE link prediction methodologies imbued with heightened efficiency and adeptness, thereby addressing salient challenges inherent to real-world applications.

show abstract

“…Networks are ubiquitous in the real world, such as social [1,2], biological [3,4], and citation [5,6] networks. If each entity in the network is regarded as a node, and the interactions between entities are regarded as edges, the relationship between entities can be visualized as graph-structured data [7,8].…”

Section: Introductionmentioning

confidence: 99%

Multi-Order-Content-Based Adaptive Graph Attention Network for Graph Node Classification

et al. 2023

View full text Add to dashboard Cite

In graph-structured data, the node content contains rich information. Therefore, how to effectively utilize the content is crucial to improve the performance of graph convolutional networks (GCNs) on various analytical tasks. However, current GCNs do not fully utilize the content, especially multi-order content. For example, graph attention networks (GATs) only focus on low-order content, while high-order content is completely ignored. To address this issue, we propose a novel graph attention network with adaptability that could fully utilize the features of multi-order content. Its core idea has the following novelties: First, we constructed a high-order content attention mechanism that could focus on high-order content to evaluate attention weights. Second, we propose a multi-order content attention mechanism that can fully utilize multi-order content, i.e., it combines the attention mechanisms of high- and low-order content. Furthermore, the mechanism has adaptability, i.e., it can perform a good trade-off between high- and low-order content according to the task requirements. Lastly, we applied this mechanism to constructing a graph attention network with structural symmetry. This mechanism could more reasonably evaluate the attention weights between nodes, thereby improving the convergence of the network. In addition, we conducted experiments on multiple datasets and compared the proposed model with state-of-the-art models in multiple dimensions. The results validate the feasibility and effectiveness of the proposed model.

show abstract

Heterogeneous deep graph convolutional network with citation relational BERT for COVID-19 inline citation recommendation

Cited by 9 publications

References 47 publications

A novel NIH research grant recommender using BERT

A novel NIH research grant recommender using BERT

Convolutional Models with Multi-Feature Fusion for Effective Link Prediction in Knowledge Graph Embedding

Multi-Order-Content-Based Adaptive Graph Attention Network for Graph Node Classification

Contact Info

Product

Resources

About