Consensus‐based journal rankings: A complementary tool for bibliometric evaluation

Abstract

Annual journal rankings are usually considered a tool for the evaluation of research and researchers. Although they are an objective resource for such evaluation, they also present drawbacks: (a) the uncertainty about the definite position of a target journal in the corresponding annual ranking when selecting a journal, and (b) in spite of the nonsignificant difference in score (for instance, impact factor) between consecutive journals in the ranking, the journals are strictly ranked and eventually placed in different terciles/quartiles, which may have a significant influence in the subsequent evaluation. In this article we present several proposals to obtain an aggregated consensus ranking as an alternative/complementary tool to standardize annual rankings. To illustrate the proposed methodology we use as a case study the Journal Citation Reports, and in particular the category of Computer Science: Artificial Intelligence (CS:AI). In the context of the consensus rankings obtained by the different methods, we discuss the convenience of using one or the other procedure according to the corresponding framework. In particular, our proposals allow us to obtain consensus rankings that avoid crisp frontiers between similarly ranked journals and consider the longitudinal/temporal evolution of the journals.

A worldwide bibliometric analysis of published literature on workplace violence in healthcare personnel

“…Bibliometrics analysis is based on the utilization of different measures/indicators [ 47 ], such as the ISI Impact Factor [ 48 ] by JCR [ 49 ] or the SJR [ 50 ] by SCImago Journal and Country Rank (SJC) [ 51 ]. In that sense, at least half of top-10 scientific journals which focus on workplace violence in healthcare personnel were in the highest quartile score (Q1) both JCR and SJR.…”

Psychology and Wikipedia: Measuring Psychology Journals’ Impact by Wikipedia Citations

“…One of the possible reasons for this phenomenon may be that the alternatives to mainstream journal rankings are not widely available. Even though some researchers discuss the possibility to rely on consensus-based evaluations (Aledo et al, 2018), and there are also some metrics that include, for example, social networks’ popularity of publications (Papakostidis & Giannoudis, 2018), they are much less universally recognized or accepted. Some scholars also consider refining methodologies of journal evaluations (Pagani et al, 2015).…”

Extensión del concepto de utopía para el problema de la agregación de rankings sin empates

“…Los elementos empatados forman un bucket. Entonces, un ranking también puede entenderse como un orden de preferencia (disjunto) entre sus buckets [18].…”