Uses of the Journal Impact Factor in national journal rankings in China and Europe

Abstract

This paper investigates different uses of the Journal Impact Factor (JIF) in national journal rankings and discusses the merits of supplementing metrics with expert assessment. Our focus is national journal rankings used as evidence to support decisions about the distribution of institutional funding or career advancement. The seven countries under comparison are: China, Denmark, Finland, Italy, Norway, Poland, and Turkey-and the region of Flanders in Belgium. With the exception of Italy, top-tier journals used in national rankings include those classified at the highest level, or according to tier, or points implemented. A total of 3,565 (75.8%) out of 4,701 unique top-tier journals were identified as having a JIF, with 55.7% belonging to the first Journal Impact Factor quartile. Journal rankings in China, Flanders, Poland, and Turkey classify journals with a JIF as being top-tier, but only when they are in the first quartile of the Average Journal Impact Factor Percentile. Journal rankings that result from expert assessment in Denmark, Finland, and Norway regularly classify journals as top-tier outside the first quartile, particularly in the social sciences and humanities. We conclude that experts, when tasked with metric-informed journal rankings, take into account quality dimensions that are not covered by JIFs.

Sorting out journals: The proliferation of journal lists in China

“…In addition, it was also confirmed in the expert review session of the Work Exchange Promotion Meeting for the China STM Journal Excellence Action Plan in April 2021, in which participating experts stressed how much the JIF increased under the Excellence Action Plan funding. Moreover, this persistence resonates with a cross-countries study (Kulczycki et al, 2022) showing that China's journal rankings are still JIF-based, and that top-tier journals explicitly refer to the first Impact Factor quartile (Q1). In the Excellence Action Plan, the funded top-tier journals mainly belong to China-owned journals in Q1.…”

“…Moreover, these lists present a tension with science policies intended to reduce over-reliance on quantitative output indicators such as the JIF. While in the journal list logic qualitative criteria can in principle displace quantitative assessment, the JIF continues to play a central role in journal assessment, which coordinates with findings from Kulczycki et al (2022). In various assessment settings, the JIF reappears indirectly as a criterion to list journals.…”

Inconsistent quality signals: evidence from the regional journals

“…Thus, one cannot expect a quick change toward the widespread publication of various journal quality statistics on journal websites and aggregators. The movement toward the construction and publication of a national expert list of high-quality journals is still in its nascent stage, but more and more countries join the discussion (Kulczycki et al, 2022 ; Pölönen et al, 2021 ).…”

Should experts guide research merit assessment? Discrepancies between Journal Impact Factor and national expert-based journal ranking list for evaluation of scientific achievement in Poland – a bibliometric analysis

“…The first model is used e.g., in Turkey or China, where journal ranking lists are based on data and metrics obtained exclusively from the Web of Science. The second model is used e.g., in Finland, Norway and Poland (6). Though both models rely to some degree on JIF, the latter is more subjective and likely to be shaped by the national science policy objectives.…”