Progressive and degenerative journals: on the growth and appraisal of knowledge in scholarly publishing

Abstract

Despite continued attention, finding adequate criteria for distinguishing "good" from "bad" scholarly journals remains an elusive goal. In this essay, I propose a solution informed by the work of Imre Lakatos and his methodology of scientific research programmes (MSRP). I begin by reviewing several notable attempts at appraising journal quality -focusing primarily on the impact factor and development of journal blacklists and whitelists. In doing so, I note their limitations and link their overarching goals to those found within the philosophy of science. I argue that Lakatos's MSRP and specifically his classifications of "progressive" and "degenerative" research programmes can be analogized and repurposed for the evaluation of scholarly journals. I argue that this alternative framework resolves some of the limitations discussed above and offers a more considered evaluation of journal quality -one that helps account for the historical evolution of journal-level publication practices and attendant contributions to the growth (or stunting) of scholarly knowledge. By doing so, the seeming problem of journal demarcation is diminished. In the process I utilize two novel tools (the mistake index and scite index) to further illustrate and operationalize aspects of the MSRP.

Exploring research quality and journal representation: A comparative study of African Journals Online, Web of Science, and Scopus

“…Journal quality is usually measured using quantitative/objective -based on citations (e.g., Garfield, 1999;Saha et al, 2003)-and qualitative/subjective criteria -based on the fulfillment of specific conditions that vary depending on the evaluator (Pölönen et al, 2021)-. Quantitative criteria have also been identified with 'research impact' rather than 'research quality', which is highly related to the internal policies of journals (Dunleavy, 2022). Qualitative criteria thus serve as guidelines to assess if a journal meets specific quality standards.…”

On the Dearth of Retractions in Social Work: A Preliminary Analysis of Ten Leading Journals

“…Grieneisen & Zhang, 2012, supplementary Table S2; Wray & Andersen, 2018); dispelling the erroneous assumption that poor quality or otherwise suboptimal scholarship is published only in low-ranking journals (see generally Dunleavy, 2021;2022c;Trikalinos et al, 2008).…”

“…Fanelli (2013) hypothesizes that these figures, rather than signifying an increase in the rate of fraud and misconduct, 3 reflect that researchers, editors, and institutions are more adept at identifying (and increasingly likely to report), papers that cause concern. Indeed, these actions may be a marker of a progressive journal or publication system (Dunleavy, 2022c), one that prioritizes knowledge-generation over non-epistemic factors, such as novelty or perceived impact. However, when retractions are scarce or altogether absent, it may signify suboptimal mechanisms for self-correction within journals or the broader scientific community (Horbach & Halffman, 2019;Ioannidis, 2012;Stroebe et al, 2012).…”

The List of Questionable Journals: Is It Trustworthy?

“…Additionally, Beall's list of predatory journals has also faced criticism due to its vague criteria (Teixeira da Silva and Kimotho, 2021). Indeed, there appears to be a gray area between "good" and "bad" (Dunleavy, 2022), which casts doubt on the credibility of the list of questionable journals (Teixeira da Silva and Tsigaris, 2018).…”