Critical Essay: Meta-analysis: A critical realist critique and alternative

Abstract

Meta-analysis has proved increasingly popular in management and organisation studies as a way of combining existing empirical quantitative research to generate a statistical estimate of how strongly variables are associated. Whilst a number of studies identify technical, procedural and practical limitations of meta-analyses, none have yet tackled the meta-theoretical flaws in this approach. We deploy critical realist meta-theory to argue that the individual quantitative studies, upon which meta-analysis relies, lack explanatory power because they are rooted in quasi-empiricist meta-theory. This problem, we argue, is carried over in meta-analyses. We then propose a 'Critical Realist Synthesis' as a potential alternative to the use of meta-analysis in organisation studies and social science more widely.

Application of Bayesian evidence synthesis to modelling the effect of ketogenic therapy on survival of high grade glioma patients

“…Furthermore, we have incorporated mechanistic reasoning and case reports through priors and shown that this improves the model fit, providing a link between explanatory qualitative research and statistical associations, both of which are required to establish causation [ 80 ]. In this way our evidence synthesis overcomes some of the limitations of conventional meta-analysis which typically excludes qualitative and experimental empirical research (such as preclinical studies and case reports) and therefore lacks explanatory power [ 81 ].…”

Examining the Relationships Between Coaching Practice and Athlete “Outcomes”: A Systematic Review and Critical Realist Critique

“…Further, it also provided a relevant platform to consider the integration of theory from these different positions and if it may be possible to conceptualise issues in an interdisciplinary manner (North, 2017;Wiltshire, 2018). To be clear, although CR may offer a useful framework to do so in future research, the aim of this paper was not to identify how and why specific coaching practice was related to particular athlete outcomes (Brannan, Fleetwood, O'Mahoney, & Vincent, 2017). Instead, the principal aim was to investigate how relationships between coaching practice and athlete outcomes have been researched to date.…”

An outsider-researcher’s tale: The paradox of studying ‘my own people’ from a peripheral position, perspective and context

“…Allen (2004) affirms that critical realists have the opportunity to reflect on their experiences and theoretical stance and consider how these affect the process and findings of the research. This was evident in my research in the way that reflexivity enabled me to consciously acknowledge my own beliefs and positioning (Brannan et al, 2017). I then used this reflexive tool to examine the principals' strategic responses to the Department of Education policies.…”