Dual Attention Model for Citation Recommendation

Abstract

Based on an exponentially increasing number of academic articles, discovering and citing comprehensive and appropriate resources has become a non-trivial task. Conventional citation recommender methods suffer from severe information loss. For example, they do not consider the section of the paper that the user is writing and for which they need to find a citation, the relatedness between the words in the local context (the text span that describes a citation), or the importance on each word from the local context. These shortcomings make such methods insufficient for recommending adequate citations to academic manuscripts. In this study, we propose a novel embedding-based neural network called "dual attention model for citation recommendation (DACR)" to recommend citations during manuscript preparation. Our method adapts embedding of three semantic information: words in the local context, structural contexts 1 , and the section on which a user is working. A neural network model is designed to maximize the similarity between the embedding of the three input (local context words, section and structural contexts) and the target citation appearing in the context. The core of the neural network model is composed of self-attention and additive attention, where the former aims to capture the relatedness between the contextual words and structural context, and the latter aims to learn the importance of them. The experiments on real-world datasets demonstrate the effectiveness of the proposed approach.

A Hybrid Approach for Paper Recommendation

“…Choi et al use deep neural networks to train the feature vectors built by title and abstract for patent citation recommendation [21]. Another study proposes a novel embedding-based neural network model for citation recommendation that captures the relatedness and importance of words in the context [22]. The study of Ali et al reviews the application of deep learning in the domain of citation recommendation [23].…”

Recommending Multiple Positive Citations for Manuscript via Content-Dependent Modeling and Multi-Positive Triplet

“…However, when applying to assist the writing of a paper, they generally lack considerations on detecting the citing intents of users. Contextbased methods [8,29] aimed to extract citing intent via an input local context (surrounding words around a target citation), and hence find the most relevant papers based on the detected citing intent. Nevertheless, their methods still leave room for further improvements.…”

“…However, Word2Vec and Doc2Vec generally treated the input documents as "plain texts" which may lead to information loss issues. Later approaches were considered to fine-tune with specific information from academic papers, such as hyperlinks [8], and section headers and word-wise relations in the local context [29]. The recent language modelling models such as BERT [5], SciBERT [2] are effective on multiple NLP tasks.…”

“…However, these methods still lack considerations on extracting the citing intent from the user's manuscript. Later researches utilized the "local context" (surrounding words around a target citation) to extract the semantics of the citing intent and then rank the most similar papers via citations embeddings [8,29].…”

“…Context-based approaches [8,29] detect citing intents from the sentences needed support, and based on which it finds the bestmatched candidate papers. These approaches could be potentially applied in the real world to assist researchers in finding citations during the writing of papers.…”