Estimating the accuracies of journal impact factor through bootstrap

Chen, Kuan-Ming; Jen, Tsung-Hau; Wu, Margaret

doi:10.1016/j.joi.2013.11.007

Cited by 7 publications

(6 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such uncertainty measures can be found in Schubert and Glänzel (1983), Nieuwenhuysen and Rousseau (1988), Opthof (1997), Chen, Jen, andWu (2014), Greenwood (2007), and Stern (2013). The latter two articles show that, besides the very top journals, a distinct discrimination in terms of the level of the impact factor between closely spaced journals is not possible.…”

mentioning

confidence: 99%

How robust are journal rankings based on the impact factor? Evidence from the economic sciences

Seiler

Wohlrabe

2014

Journal of Informetrics

View full text Add to dashboard Cite

Standard-Nutzungsbedingungen:Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden.Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen.Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. It is well-known that the distribution of citations to articles in a journal is skewed. We ask whether journal rankings based on the impact factor are robust with respect to this fact. We exclude the most cited paper, the top 5 and 10 cited papers for 100 economics journals and recalculate the impact factor. Afterwards we compare the resulting rankings with the original ones from 2012. Our results show that the rankings are relatively robust. This holds both for the 2-year and the 5-year impact factor. Terms of use: Documents in EconStor mayJEL-Code: A120, A140.

show abstract

mentioning

confidence: 99%

How robust are journal rankings based on the impact factor? Evidence from the economic sciences

Seiler

Wohlrabe

2014

Journal of Informetrics

View full text Add to dashboard Cite

show abstract

“…Such is the case when journal impact factors are used to create journal rankings (for example the SCImago Journal Rank). Journal impact factors are also frequently the object of ensuing measurements and calculations concerning questions such as the precision and reliability of such metrics (for example Chen, Jen & Wu, 2014;Donner, 2018;Moed et al, 1996;Van Leeuwen & Moed, 2005). In this case, the value-measurement links progressively lose sight of their original artefact and substitute it with other value-measurement links.…”

Section: Unfolding Chains Of Evaluationmentioning

confidence: 99%

Evaluations as value-measurement links: Exploring metrics and meanings in science

Hesselmann

Schendzielorz

2019

Social Science Information

View full text Add to dashboard Cite

This contribution seeks to provide a more detailed insight into the entanglement of value and measurement. Drawing on insights from semiotics and a Bourdieusian perspective on language as an economy of linguistic exchange, we develop the theoretical concept of value-measurement links and distinguish three processes – operationalisation, nomination, and indetermination – as forms in which these links can be constructed. We illustrate these three processes using (e)valuation practices in science, particularly the journal impact factor, as an empirical object of investigation. As this example illustrates, measured values can function as building blocks for further measurements, and thus establish chains of evaluations, where it becomes more and more obscure which values the measurements actually express. We conclude that in the case of measured values such as impact factors, these chains are driven by the interplay between the interpretative openness of language and the seeming tendency of numbers to fixate meaning thus continually re-creating, transforming and modifying values.

show abstract

“…Leydesdorff and Opthof (2010), Moed et al (2012), and Vanclay (2012) called for confidence intervals to be provided for journal metrics. Such uncertainty measures can be found, for instance, in Schubert and Glänzel (1983), Nieuwenhuysen and Rousseau (1988), Opthof (1997), Greenwood (2007), Stern (2013), and Chen et al (2014). Therefore, we decided to base our decision for a draw on official stability intervals, provided by CWTS Journal Indicators.…”

Section: The Elo Rating Systemmentioning

confidence: 99%

Who is the ‘Journal Grand Master’? A new ranking based on the Elo rating system

Lehmann

Wohlrabe

2017

Journal of Informetrics

View full text Add to dashboard Cite

Abstract. In this paper we transfer the Elo rating system, which is widely accepted in chess, sports and other disciplines, to rank scientific journals. The advantage of the Elo system is the explicit consideration of the factor time and the history of a journal's ranking performance. Most other rankings that are commonly applied neglect this fact. The Elo ranking methodology can easily be applied to any metric, published on a regular basis, to rank journals. We illustrate the approach using the SNIP indicator based on citation data from Scopus. Our data set consists of more than 20 000 journals from many scientific fields for the period from 1999 to 2015. We show that the Elo approach produces similar but by no means identical rankings compared to other rankings based on the SNIP alone or the Tournament Method. Especially the rank order for rather 'middle-class' journals can tremendously change.

show abstract

Estimating the accuracies of journal impact factor through bootstrap

Cited by 7 publications

References 30 publications

How robust are journal rankings based on the impact factor? Evidence from the economic sciences

How robust are journal rankings based on the impact factor? Evidence from the economic sciences

Evaluations as value-measurement links: Exploring metrics and meanings in science

Who is the ‘Journal Grand Master’? A new ranking based on the Elo rating system

Contact Info

Product

Resources

About