Studying review articles in scientometrics and beyond: a research agenda

Abstract

Review articles are an often neglected genre in scholarly communication. Though there was intense discussion about review articles in scientometrics in the 1970s and 1980s, we find less studies devoted to this genre within the last 20 years. Yet, recent discussions in other fields, such as linguistics, sociology or medicine imply that review articles are part of important debates about problems of research in academia, such as research quality or transparency. Against that background, the purpose of this paper is to review recent developments for the study of review articles in scientometrics and beyond, to discuss theoretical, conceptual and empirical accounts of how review articles can be defined, and to identify major methodological and conceptual challenges for studying review articles. Based on reviewing work and inputs received from of a workshop conducted at a Conference of the International Society of Informetrics in September 2019, we propose a research agenda for the study of review articles. We have identified six realms of study in this area: (1) the study of methodological caveats resulting from the usage of scholarly databases, (2) the study of field specific patterns of reception and usage of review articles, (3) the study of argumentative and textual structures of review articles, (4) the exploration of organizations and infrastructures for review articles, (5) the study of epistemic roles of review articles, and (6) the analysis of authorship patterns in review articles.

A comparison of systematic reviews and guideline-based systematic reviews in medical studies

“…Other countries with high rates of these genres, most notably the Netherlands or Denmark, are also among the top ten adopters to evidence-based medicine (ebd.). Although the "evidence-based" movement promotes and comes with a variety of knowledge generation practices, systematic reviewing remains an important cornerstone in EBM as other studies have indicated (Blümel & Schniedermann, 2020;Ojasoo et al, 2001). In that sense, the data provided here corresponds with other studies that mentioned the link between evidence-based paradigms and the emergence of systematic reviews (Timmermans & Berg, 2003;Straßheim and Kettunen 2014).…”

“…Similar dynamics have also been observed for academic communities consenting on theories or methods (cf. Luetge, 2004;Zollman, 2007), or the role of reviews in the formation of new disciplines (Blümel, 2020).…”

“…As such, this genre is situated at the intersection of what is perceived as high levels of epistemic quality, and what is actually measured by the wide palette of bibliometric indicators. For example, because review articles generally accumulate higher citation counts than reports of primary researchan observation long known to bibliometricians-they are suspected to be used strategically by some academic journals and individual researchers (Blümel & Schniedermann, 2020). Some bibliometric assessments found that systematic reviews and meta-analyses inherit this characteristic and are also used strategically by impact-sensitive actors (Barrios et al, 2013;Colebunders et al, 2014;Ioannidis, 2016;Patsopoulos et al, 2005;Royle et al, 2013).…”

A Bibliometric analysis of folate receptor research

“…Moreover, our study detected an appreciable number of review articles (426 total) in addition to primary research articles that could be analyzed in future studies. A recent report by Blumel and Shniederman, based on inputs from the Conference of the International Society of Informetrics [ 76 ], proposes a broad agenda for the study of review articles. However, including such sizable numbers review articles within the framework of our current analyses would have been problematic.…”

Lone Geniuses or One among Many? An Explorative Study of Contemporary Highly Cited Researchers

“…In the WoS database, each item is classified into a particular "document type" category, where "review" is one. Although there is a lack of a commonly accepted definitions of review articles, the classification applied in the WoS database has been proven to be inaccurate (Blumel & Schniedermann, 2020;Harzing, 2013). In particular, this refers to a criterion where all items containing over 100 references are automatically classified as review papers.…”