Comparing paper ranking algorithms

Abstract

The research presented in this paper focuses on comparing and evaluating various ranking algorithms that can be used on citation graphs in order to rank individual papers according to their importance and relevance. The graph analysis algorithms investigated in this paper are PageRank, CiteRank and an algorithm proposed by Hwang et al.[9] and compared to the method of simply counting the number of citations of a publication. In addition, a new algorithm, NewRank, is proposed which is a combination of the PageRank and CiteRank algorithms with the focus on identifying influential papers that were published recently. A customizable crawler framework was developed to collect publication datasets from various sources. The development of this framework is discussed in detail. Finally, the ranking algorithms are evaluated against the list of the most influential papers compiled by the ICSE selection committee 1 .

Identification of milestone papers through time-balanced network centrality

“…In principle, one might consider a comparison of the final ranking positions (i.e., the ranking positions computed on the whole dataset) of the target papers by a certain metric [18,21] instead of the age-dependent evaluation of the metrics introduced above. But this kind of comparison would miss our key point -the strong dependence of metrics' performance on paper age.…”

“…Although some effort has been devoted to contrast different metrics with respect to their ability to single out seminal papers [18][19][20][21], differences among the adopted benchmarking procedures and diverse conclusions of the mentioned references leave a fundamental question still open: which metric of scientific impact best agrees with expert-based perception of significance? In agreement with ref.…”

Feature-enriched author ranking in incomplete networks

“…1). On one hand, paper-level ranking uses the papers' citation network to diffuse scientific credit to cited papers, and then authors credit is derived from the credit of their papers (Hwang et al 2010;Dunaiski and Visser 2012). On the other hand, author-level ranking uses the authors' citation network to diffuse scientific credit to cited authors, thus the authors' credit is directly obtained (Radicchi et al 2009;Ding 2009;West et al 2013).…”

“…Regarding edges, in paper-level networks edges are traditionally unweighted and simple, i.e., two papers are connected by a single edge with weight equal to 1 (Hwang et al 2010;Dunaiski and Visser 2012). In author-level networks, edges are weighted and multiple, i.e., two authors are connected by multiple edges with different weights.…”

“…For example, if A cites B, and B cites C, traditional bibliometrics give no additional credit to C from A's indirect citation. To address this limitation, graph algorithms have been developed for citation networks (Ding 2009;Radicchi et al 2009;Dunaiski and Visser 2012;West et al 2013;Sidiropoulos and Manolopoulos 2006). These algorithms are modifications of PageRank applied to citation networks (Page et al 1998).…”