Achieving Rigor in Literature Reviews: Insights from Qualitative Data Analysis and Tool-Support

Abstract

It is important for researchers to efficiently conduct quality literature studies. Hence, a structured and efficient approach is essential. We overview work that has demonstrated the potential for using software tools in literature reviews. We highlight the untapped opportunities in using an end-to-end tool-supported literature review methodology. Qualitative data-analysis tools such as NVivo are immensely useful as a means to analyze, synthesize, and write up literature reviews. In this paper, we describe how to organize and prepare papers for analysis and provide detailed guidelines for actually coding and analyzing papers, including detailed illustrative strategies to effectively write up and present the results. We present a detailed case study as an illustrative example of the proposed approach put into practice. We discuss the means, value, and also pitfalls of applying tool-supported literature review approaches. We contribute to the literature by proposing a four-phased tool-supported methodology that serves as best practice in conducting literature reviews in IS. By viewing the literature review process as a qualitative study and treating the literature as the "data set", we address the complex puzzle of how best to extract relevant literature and justify its scope, relevance, and quality. We provide systematic guidelines for novice IS researchers seeking to conduct a robust literature review.

Which academic search systems are suitable for systematic reviews or meta‐analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources

“…Documentation of the search process “makes the search replicable and provides a clear starting point for later updates.” Building a systematic review team that incorporates diverse expertise in areas such as content, systematic review methods, searching, and quantitative synthesis has been shown to significantly improve the quality of the review work . Building on an understanding of constructs, a systematic search is the gatekeeper that establishes the basis for subsequent synthesis . This process defines the scope of the examination and thus influences the outcome of the analysis.…”

“…9 Building on an understanding of constructs, a systematic search is the gatekeeper that establishes the basis for subsequent synthesis. 10 This process defines the scope of the examination and thus influences the outcome of the analysis. The same analysis employed with different samples might lead to different results.…”

The Unfolding and Resurfacing of Information Systems Knowledge over the Last 25 Years: A Systemic Perspective

“…In our study, we adopt the Bandara et al . () method for IS literature reviews. This is one among many possible ones.…”

“…The Bandara et al . () method suggests four stages: (i) extraction of relevant IS literature, (ii) organization and preparation for analysis, (iii) coding (including mapping of associations between concepts using Leximancer ) to support iterative analysis and (iv) write up and presentation.…”

“…Using a software tool ( Leximancer ®, for this study we chose Leximancer® Version 4.0 http://info.leximancer.com/), we adapt the method of analysis of IS literature by Bandara et al . () to identify relevant concepts, themes and associations between them and elicit different stages of knowledge of dominance and eclecticism in a period of 25 years and using 4100 abstracts in eight prominent IS journals. Although our study does not intend to provide comprehensive syntheses or theoretical contributions about IS knowledge, the findings suggest that IS key concepts like systems , information and use reconfigure as well as resurface over time, helping the field keep a healthy balance and mutual dependence between dominance and eclecticism.…”