Generalising from qualitative evaluation

Abstract

Evaluations are often focused on assessing merit, value, outcome or some other feature of a programme, project, policy or some other object. Evaluation research is then more concerned with the particular rather than the general – even more so, when qualitative methods are used. But does this mean that evaluations should not be used to generalise? If it is possible to generalise from evaluations, under what circumstances can this be legitimately achieved? The authors of this article have previously argued for generalising from qualitative research (GQR), and in this article, they extrapolate the discussion to the field of evaluation. First, the article begins with a discussion of the definitions of generalisability in research, recapping briefly on our arguments for GQR. Second, the differentiation between research and evaluation is explored with consideration of what literature there is to justify generalisation from qualitative evaluation (GQE). Third, a typology derived from the literature is developed, to sort 54 evaluation projects. Fourth, material from a suite of evaluation projects is drawn from to demonstrate how the typology of generalisation applies in the context of evaluations conducted in several fields of study. Finally, we suggest a model for GQE.

Analysing a Transnational Advocacy Network of Internet Data Affordability in Malawi

“…Similar studies can be conducted in other contexts and compare the findings. Hence, the findings from the study showed analytical generalisations rather than statistical generalisations (Guenther & Falk, 2021).…”

Advancing the Impact of Critical Qualitative Research on Policy, Practice, and Science

“…They propose that, against conventional notions of analysis as a linear application of method, "thinking with theory does not come at the end of anything but is emergent and immanent to that which is becoming" (p. 1232). In this way, thinking with theory is a process that informs an analytic endeavor from the beginning, shaping the ways in which the world is experienced, data are generated, and sense is made (Guenther & Falk, 2021). Jackson and Mazzei (2018) outline the value of drawing on concepts from theories of various kinds to pursue "eruptions", which are analytic moves that push thinking beyond taken-forgranted conceptualizations of the topic of a given study.…”

“…They propose that, against conventional notions of analysis as a linear application of method, “thinking with theory does not come at the end of anything but is emergent and immanent to that which is becoming” (p. 1232). In this way, thinking with theory is a process that informs an analytic endeavor from the beginning, shaping the ways in which the world is experienced, data are generated, and sense is made (Guenther & Falk, 2021).…”

Theory building from qualitative evaluation

“…There is a related/parallel argument that suggests evaluation studies can result in generalisation (Guenther and Falk, 2021) through iterative processes of theory building and testing (Guenther and Falk, 2019). Mark (2011), in his discussion about generalisability, calls for more attention to be given to 'translational theories ' (p. 38), leading to prediction based on what he describes as 'principled discovery' (p. 40).…”

“…The interest in qualitative evaluation arises from the authors' use of qualitative and mixed methodologies in research and evaluation. The work here builds on previous work published in relation to generalising from qualitative research (Falk and Guenther, 2021;Guenther and Falk, 2019) and evaluation (Guenther and Falk, 2021).…”