Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations

Abstract

The Journal Impact Factor (JIF) was originally designed to aid libraries in deciding which journals to index and purchase for their collections. Over the past few decades, however, it has become a relied upon metric used to evaluate research articles based on journal rank. Surveyed faculty often report feeling pressure to publish in journals with high JIFs and mention reliance on the JIF as one problem with current academic evaluation systems. While faculty reports are useful, information is lacking on how often and in what ways the JIF is currently used for review, promotion, and tenure (RPT). We therefore collected and analyzed RPT documents from a representative sample of 129 universities from the United States and Canada and 381 of their academic units. We found that 40% of doctoral, research-intensive (R-type) institutions and 18% of master’s, or comprehensive (M-type) institutions explicitly mentioned the JIF, or closely related terms, in their RPT documents. Undergraduate, or baccalaureate (B-type) institutions did not mention it at all. A detailed reading of these documents suggests that institutions may also be using a variety of terms to indirectly refer to the JIF. Our qualitative analysis shows that 87% of the institutions that mentioned the JIF supported the metric’s use in at least one of their RPT documents, while 13% of institutions expressed caution about the JIF’s use in evaluations. None of the RPT documents we analyzed heavily criticized the JIF or prohibited its use in evaluations. Of the institutions that mentioned the JIF, 63% associated it with quality, 40% with impact, importance, or significance, and 20% with prestige, reputation, or status. In sum, our results show that the use of the JIF is encouraged in RPT evaluations, especially at research-intensive universities, and indicates there is work to be done to improve evaluation processes to avoid the potential misuse of metrics like the JIF.

Ten hot topics around scholarly publishing

“…С этого момента ИФЖ стал ассоциироваться со знаком качества журнала и широко использоваться для оценки исследований и исследователей, даже на институциональном уровне. Таким образом, он оказывает значительное влияние на методы и проведение исследований в области управления [16][17][18].…”

“…Несмотря на неуместность ИФЖ, многие страны регулярно используют его для оценки исследования [18,30,31], что создает двухуровневую систему подсчета баллов, которая автоматически приписывает более высокие баллы (например, тип А) статьям, опубликованным в журналах с ИФ или журналах, индексируемых в международных БД, и более низкий результат (например, тип Б) статьям, опубликованным в местных журналах. Совсем недавно организация, которая формально рассчитывает ИФЖ, выпустила отчет с описанием его сомнительного использования 10 .…”

“…Подобные статьи с участием профессоров университетов появились в десятках журналов 17 . Недавние судебные документы показали, что Monsanto написал такие статьи, чтобы противостоять правительственной оценке канцерогенности пестицида глифосата и воздействовать на Международное агентство по исследованиям рака 18 . Таким образом, рецензирование представляется в значительной степени неадекватным для разоблачения или выявления конфликтов интересов и смягчения потенциального воздействия.…”

A tale of two 'opens': intersections between Free and Open Source Software and Open Scholarship

“…Present evaluation systems are dominated by metrics based on citation counts (Wagner 2016;McKiernan et al 2019). Citation, in its rawest form, is usually an acknowledgement of an intellectual debt (Garfield 1979), and a hierarchical genealogy and network of ideas, dependent on a number of factors.…”

“…Bostrom 2017), and represent major challenges in defining 'value' in [open] scholarly research.At the present, few of the things which we might use to define 'better' scholarship(whether 'open' or not) are what we seem to value in terms of how research is evaluated or rewardedSchimanski and Alperin 2018;McKiernan et al 2019). The former concept is based around things such as sharing of knowledge, ideas, and outputs, rigorous and reproducible research, and…”