hyperdoc2vec: Distributed Representations of Hypertext Documents

Abstract

Hypertext documents, such as web pages and academic papers, are of great importance in delivering information in our daily life. Although being effective on plain documents, conventional text embedding methods suffer from information loss if directly adapted to hyper-documents. In this paper, we propose a general embedding approach for hyper-documents, namely, hyperdoc2vec, along with four criteria characterizing necessary information that hyper-document embedding models should preserve. Systematic comparisons are conducted between hyperdoc2vec and several competitors on two tasks, i.e., paper classification and citation recommendation, in the academic paper domain. Analyses and experiments both validate the superiority of hyperdoc2vec to other models w.r.t. the four criteria.

Citation recommendation: approaches and datasets

“…Moreover, high thresholds for the translation probability may be set to make the machine translation approach feasible [74]. (d) Neural networks [48,67,75,91,153,174] This group contains not only many approaches to local citation recommendation (6 out of 17, that is, 35%), but also the most recent ones: here, papers have been published since 2014. Due to the large field of neural network research in general, the architectures proposed here also vary considerably.…”

Citation Recommendations Considering Content and Structural Context Embedding

“…Given a citation relation, DocCit2Vec picks one publication from the citation list as the target, and utilizes the surrounding context and structural context as known knowledge to maximize the occurrence of the target citation by updating the parameters (i.e., the embedding vectors) of the neural network. The model learns two document embedding vectors IN and OUT, where the IN vector d I characterizes the document as a citing paper and the OUT vector d O encodes its role as a cited paper [9]. In addition, the model learns IN word vectors w I .…”

“…The architecture of DocCit2Vec-avg is constructed based on the pv-dm structure of Doc2Vec [14] and HyperDoc2Vec [9], shown on the right side of {w|w ∈ C }. The output layer is computed using a multiclass softmax classifier, and the output value is regarded as the probability of the occurrence of d t .…”

“…Two pre-processed datasets DBLP and ACL Anthology [9], containing full texts of academic papers in the computer science domain are utilized. Statistical summaries of the datasets are provided in Table I.…”

“…We adopt different similarity calculation methods for the best performance of each model, as described in [9]. We employ the IN-for-OUT (I4O) method for Word2Vec (W2V) and HyperDoc2Vec (HD2V), which uses the averaged IN feature vectors to rank the OUT document vectors by the In addition, as in [9], we run Doc2Vec-nc (D2V-nc) on the training file without citations, and Doc2Vec-cacNev (D2V-cac) on the training file with "augmented contexts", i.e., each citation context is copied into the target document. For DocCit2Vec-avg (DC2V-avg) and DocCit2Vec-att (DC2V-att), we employ the same I4O method.…”