Evaluation of untrustworthy journals: Transition from formal criteria to a complex view

Abstract

Not all the journals included in credible indices meet the ethical rules of COPE, DOAJ, OASPA, and WAME, and there may also be trustworthy journals excluded from these indices, which means they cannot be used as whitelists for trustworthy journals. Equally, the many methods suggested to determine trustworthiness are not reliable because they include questionable criteria. The question arises whether valid criteria for identifying an untrustworthy journal can be determined and whether other assessment procedures are necessary. Since 2017, the Masaryk University Campus Library has been developing a suitable evaluation method for journals. A list of 19 criteria based on those originally suggested by Beall, COPE, DOAJ, OASPA, and WAME were reduced to 10 objectively verifiable criteria following two workshops with librarians. An evaluation of 259 biomedical journals using both the list of 19 and then 10 criteria revealed that 74 journals may have been incorrectly assessed as untrustworthy using the longer list. The most common reason for failure to comply was in the provision of sufficient editorial information and declaration of article processing charges. However, our investigation revealed that no criteria can reliably identify predatory journals. Therefore, a complex evaluation is needed combining objectively verifiable criteria with analysis of a journal's content and knowledge of the journal's background.

Authors publishing repeatedly in predatory journals: An analysis of Scopus articles

“…Identifying the predatory publication outlets can be very difficult and existing frameworks struggle as they typically include problematic control criteria (Kratochvíl et al, 2020). A wide variety of degrees, types, and combinations of illegitimate behaviour (Siler, 2020) exists.…”

A credit-like rating system to determine the legitimacy of scientific journals and publishers

“…The downstream impacts of poorly developed policies, when implemented at a national scale, may hurt many academics by providing poor advice, e.g., in India (Patwardhan et al, 2018 ). When such variable, fluctuating, and to some extent unreliable (Kratochvil et al, 2020 ) lists or criteria are used to classify academics or academic institutions, there is the risk or misclassification (e.g., calling a surgery journal that may be blacklisted as a “predatory surgery journal”) (Teixeira da Silva, 2021b ), which may cause inaccurate and unfair reputational damage through false accusations or unsubstantiated mischaracterizations (Tsigaris & Teixeira da Silva, 2019 , 2020 ). Moreover, such issues and limitations are not limited to open access (Olivarez et al, 2018 ).…”

“…For this reason, Kratochvíl et al ( 2020 ) proposed to move from a simple set of formal criteria to a complex view of scholarly publishing. After assessing different lists of criteria taken from Beall, COPE, DOAJ, OASPA, and WAME, they revealed that 28% of the journals which were evaluated had been incorrectly assessed as untrustworthy.…”

Sociality and the Need for the Epistemology of Social Relatedness: IJSS Editorial Statement

“…The quality of any academic journal is dependent on its editorial process, the people involved, its originality, management of ethics issues, conflict of interest and peer-review process (Jefferson et al, 2002;Kratochvíl et al, 2020). Based on this, ERRCDF, through IJSS, has adopted a rigorous and academic tasking editorial process discussed below.…”