Meta-analysis in information systems research: method choices and recommendations for future research

Abstract

PurposeThe use of meta-analysis in information systems (IS) research has increased. Because this research domain has matured and an increasing number of empirical studies have become available, meta-analysis is now perceived as a useful approach to synthesize an increasing body of empirical research. The present paper gives an overview of meta-analytical techniques, the evolution of meta-analysis in IS, the focus of these studies and method choices made by authors.Design/methodology/approachThis overview is based on 100 meta-analysis studies published in IS journals. These meta-analytical studies synthesize data from 6,262 empirical studies, and the meta-analyses were published between 1989 and 2020. The studies were coded with regard to 16 major method choices that IS researchers made when conducting meta-analyses.FindingsThis overview describes the key areas of meta-analysis (e.g. strategy and firm performance) and the journals that publish the meta-analysis. This overview also identifies method issues where meta-analyses have improved in recent years as well as issues that require some attention. Scholars receive guidance about good practices in terms of 16 major method choices related to (1) problem formulation, (2) literature search, (3) coding information, (4) analyzing and integrating effect sizes and (5) interpreting results and reporting findings.Originality/valueAddressing the identified method issues helps authors of future meta-analyses to increase the transparency of reporting and the likelihood of getting published. The substantive findings of a meta-analysis also often depend on researchers' method choices. Similar overviews exist for other disciplines, such as marketing and management.

Relationship between teacher empowerment and job satisfaction: A Meta-Analytic path analysis

“…Another challenge was that the meta-analysis was limited to the statistics provided by the primary researchers (Blut, 2020). Primary studies reporting inadequate statistics cannot be included in a MASEM, rendering a considerable loss of primary information.…”

Knowledge management technologies and organizational performance: a meta-analytic study

“…, 2008). Widely adopted in scientific research, such as in information system research (Blut, 2021), this approach can be used to draw conclusions based on numerous studies that examine identical issues by correcting errors and biases and can reveal knowledge by investigating the characteristics of the individual studies, such as through sub-group analysis (Noel and Todd, 2012). As we aim to investigate the KMT–organizational performance relationship across empirical papers, a meta-analytic method was applied to synthesize previous scholarly findings.…”

“…Research method and implementation 3.1 Meta-analysis Meta-analysis is a methodological and statistical method (Noel and Todd, 2012) intended to produce empirical knowledge about general associations, particularly causal relationships (Matt and Cook, 2009), by statistically analyzing a large number of quantitative findings from separate studies to create conclusive generalizations (Hempel, 2020;Hartung et al, 2008). Widely adopted in scientific research, such as in information system research (Blut, 2021), this approach can be used to draw conclusions based on numerous studies that examine identical issues by correcting errors and biases and can reveal knowledge by investigating the characteristics of the individual studies, such as through sub-group analysis (Noel and Todd, 2012). As we aim to investigate the KMT-organizational performance relationship across empirical papers, a meta-analytic method was applied to synthesize previous scholarly findings.…”

Identifying high quality ideas in the online context: evidence from a meta-analysis

“…Accordingly, this study employed meta-analytic techniques that are especially suited to solving controversial research objectives (King and He, 2005; Geyskens et al. , 2009; Markus, 2020).…”

“…Accordingly, this study employed meta-analytic techniques that are especially suited to solving controversial research objectives (King and He, 2005;Geyskens et al, 2009;Markus, 2020). The meta-analysis method enabled us to combine multiple single studies into an integrated one, and thus we were able to obtain a more accurate and robust result (Lipsey and Wilson, 2001;Schmidt and Hunter, 2014).…”