Blacklists and Whitelists To Tackle Predatory Publishing: a Cross-Sectional Comparison and Thematic Analysis

Abstract

We aimed to develop an in-depth understanding of quality criteria for scholarly journals by analyzing journals and publishers indexed in blacklists of predatory journals and whitelists of legitimate journals and the lists’ inclusion criteria. To quantify content overlaps between blacklists and whitelists, we employed the Jaro-Winkler string metric. To identify topics addressed by the lists’ inclusion criteria and to derive their concepts, we conducted qualitative coding. We included two blacklists (Beall’s and Cabells Scholarly Analytics’) and two whitelists (the Directory of Open Access Journals’ and Cabells Scholarly Analytics’). The number of journals per list ranged from 1,404 to 12,357, and the number of publishers ranged from 473 to 5,638. Seventy-two journals and 42 publishers were included in both a blacklist and a whitelist. Seven themes were identified in the inclusion criteria: (i) peer review; (ii) editorial services; (iii) policy; (iv) business practices; (v) publishing, archiving, and access; (vi) website; and (vii) indexing and metrics. Business practices accounted for almost half of the blacklists’ criteria, whereas whitelists gave more emphasis to criteria related to policy. Criteria could be allocated to four concepts: (i) transparency, (ii) ethics, (iii) professional standards, and (iv) peer review and other services. Whitelists gave most weight to transparency. Blacklists focused on ethics and professional standards. Whitelist criteria were easier to verify than those used in blacklists. Both types gave little emphasis to quality of peer review. Overall, the results show that there is overlap of journals and publishers between blacklists and whitelists. Lists differ in their criteria for quality and the weight given to different dimensions of quality. Aspects that are central but difficult to verify receive little attention. IMPORTANCE Predatory journals are spurious scientific outlets that charge fees for editorial and publishing services that they do not provide. Their lack of quality assurance of published articles increases the risk that unreliable research is published and thus jeopardizes the integrity and credibility of research as a whole. There is increasing awareness of the risks associated with predatory publishing, but efforts to address this situation are hampered by the lack of a clear definition of predatory outlets. Blacklists of predatory journals and whitelists of legitimate journals have been developed but not comprehensively examined. By systematically analyzing these lists, this study provides insights into their utility and delineates the different notions of quality and legitimacy in scholarly publishing used. This study contributes to a better understanding of the relevant concepts and provides a starting point for the development of a robust definition of predatory journals.

Who reviews for predatory journals? A study on reviewer characteristics

“…Second, lists of journals are difficult to keep up do date and might not reflect recent changes in publishing practices of journals. Cabell's lists have further been shown to neglect important but challenging to verify criteria, including peer review and editorial services [7]. We hence might have included false positives and excluded false negatives from our analysis.…”

“…Cabell's is a scholarly services company that maintains a list of likely legitimate journals and a list of journals "as potentially not following scientific publication standards or expectations on quality, peer reviewing, or proper metrics" [19]. To be indexed in either list, a journal must fulfill several requirements that cover different aspects, including business practices, editorial services, publishing policies, and archiving and access [7]. Both Cabell's lists are subscription-based.…”

“…In a second step, we matched both samples with the Publons entire database of review reports, to identify reviews for predatory journals and reviews for legitimate journals. Following an approach used by Strinzel et al (2019), this matching was based on journal names [7]. Because of potential typos and orthographical differences between the Cabell's lists and the Publons database, we matched journal names based on their relative similarity, using the Jaro-Winkler algorithm in R package RecordLinkage [7,22].…”

“…Following an approach used by Strinzel et al (2019), this matching was based on journal names [7]. Because of potential typos and orthographical differences between the Cabell's lists and the Publons database, we matched journal names based on their relative similarity, using the Jaro-Winkler algorithm in R package RecordLinkage [7,22]. The Jaro-Winkler metric, which ranges from 0 (no similarity) to 1 (exact match), was calculated for all possible pairs of journals [23].…”

“…Prioritizing self-interests at the expense of scholarship, these outlets exploit the open access (OA) model of publishing. They offer to publish articles but do not provide the services one would expect to receive from a legitimate journal [5][6][7]. Amongst other deficiencies, predatory journals do not guarantee archiving and long-term access to their contents [6,8].…”