Debating Systematic Literature Reviews (SLR) and their Ramifications for IS: A rejoinder to Mike Chiasson, Briony Oates, Ulrike Schultze, and Richard Watson

Abstract

Systematic Literature Reviews (SLRs) are making their inroads into the IS discipline. By responding to the commentaries this rejoinder contributes to a debate about potential merits, limitations and wider ramifications of SLR for the IS discipline. More specifically we engage with the questions: What is an SLR and can it be conducted partially? How can literature reviews and SLR be improved? What is the view of 'evidence' in SLRs and the evidence-based practice movement and what are potential implications for 'research informing IS practice'? How can the efficiency and effectiveness of literature reviews be improved? Overall we argue for a practice of critique that scrutinizes methodologies such as SLR in the light of values and ends we seek to achieve in the IS discipline.

A typology of systematic reviews for synthesising evidence on health care

“…The term “meta‐narrative review” was coined for a similar umbrella approach to mapping and synthesising information from different sources, perspectives and research traditions . When used to search, interpret and synthesise theories, umbrella reviews use an inductive process to locate similar propositions within multiple theories or to construct a metatheory . Reviews of theory can also be considered as part of a realist review.…”

“…Modifications of the PICO, such as the “Population, phenomena of Interest and Context” (PICo) or the “Context, How, Issues, and Population” (CHIP) guides are available for qualitative and umbrella reviews; while the “Behaviour of interest; Health context; Exclusions; Models or Theories (BeHEMoTh) framework is available specifically for seeking theories . They all tend to produce high yields of irrelevant literature probably because primary qualitative studies are not reliably indexed .…”

“…16 It is criticised for being unscientific, unsystematic, unstructured, biased, nontransparent and inefficient. 22,35,36 However, the criticisms are challenged by clinicians, specialists in information systems, and others who doubt the feasibility or efficiency of comprehensive unbiased searches by whatever means, 24,[37][38][39][40] or indeed the practical relevance of controlled trials to most problems of healthcare and clinical practice. 4,23,41 At very least, the narrative review is a sources of valuable expert opinion when data from controlled trials are undeveloped, 3 but beyond that it can expose opinions, assumptions and narratives from patients, clinicians and others through insightful and helpful interpretations of responses to illness and therapies.…”

“…63 When used to search, interpret and synthesise theories, umbrella reviews use an inductive process to locate similar propositions within multiple theories or to construct a metatheory. 40,[64][65][66][67] Reviews of theory can also be considered as part of a realist review.…”

Literature Reviews in IS Research: What Can Be Learnt from the Past and Other Fields?

“…Among the top 40 IS journals that Lowry et al (2013) identify, 17 journals explicitly welcome LRs as a research genre in their editorial statements and 36 journals have published at least one LR between 2000 and 2014 (Wagner, Prester, Roche, Benlian, & Schryen, 2016). MIS Quarterly provided the opportunity to publish "theory and review" papers in 1999 (Markus & Saunders, 2007;Watson, 2001), the European Journal of Information Systems recognized the need for stronger support of LRs (Rowe, 2012(Rowe, , 2014, Communications of the AIS published a special issue on LRs in 2015 (Tate, Furtmueller, Evermann, & Bandara, 2015), and the Journal of Information Technology recently published a debate on systematic LRs (Boell & Cecez-Kecmanovic, 2015a, 2015bChiasson, 2015;Oates, 2015;Schultze, 2015;Watson, 2015). IS authors have responded to the call for LRs by contributing more than 200 LRs to the above-mentioned set of top IS journals since 2000 (Paré, Trudel, Jaana, & Kitsiou, 2015;Wagner et al, 2016).…”