Depicting the logic of three evaluation theories

Abstract

The work presented in this paper is motivated by a desire to understand the similarities and differences between various evaluation approaches or theories. As previously argued by Shadish (1998), we believe that such an understanding has value for practitioners, theorists, and those who study evaluation. Our basic premise is that visual depictions of such approaches may help to clarify their most important features, in the same way that logic models are frequently used to explicate program theories. Once the salient features are identified, it becomes more evident what particular combination of evaluator and stakeholder activities constitutes the approach. In addition, an underlying logic is revealed, including an explanation of the ways in which those activities might relate to particular effects or consequences. This may help practitioners who, for example, desire to follow a particular theorist's prescriptions. Miller (2010) has suggested that one way in which we might assess the quality or usefulness of evaluation theories is on the basis of their operational specificity. It is well known that constructing a program logic model can sometimes reveal a less-than-fully-formed program logic. There may be similar value in depicting evaluation theories, as such efforts could help identify those aspects of a theory that are in need of further development. Finally, we believe that creating these visual representations may aid research on evaluation. The models we present here include suggestions and claims about the settings or circumstances in which an approach might be followed, the activities that should be implemented, and the effects that may be expected. From these suggestions and claims, it is possible to identify countless questions that could be examined in future empirical research. Indeed, comparing the existing research base with the assertions made in these models could inform the development of a research agenda, in which certain questions are prioritized on the basis of their relevance to theories of practice. This would allow these theories of evaluation to be tested and, over time, refined. Background A (borrowed) framework for modeling evaluation theoriesAs an organizing framework for our visual depictions of the three evaluation theories, we adapted a logic model template developed by the Program Development and Evaluation Unit at the University of Wisconsin-Extension (UWEX). Details of this model are described in a logic model training manual by Taylor-Powell and Henert (2008). Its core elements-including activities and outcomes-resemble those of other common logic model frameworks Evaluation and Program Planning 38 (2013) Here, we describe the development of logic models depicting three theories of evaluation practice: Practical Participatory , Values-engaged (Greene, 2005a, and Emergent Realist . We begin with a discussion of evaluation theory and the particular theories that were chosen for our analysis. We then outline the steps involved in constructing the models. The theoretical prescript...

Logic models: A useful way to study theories of evaluation practice?

“…Through the logic modeling analyses presented in these papers we see clearly how each of these evaluation approaches conforms to one or more of these theoretical aspects. For example, Hansen et al (2013) illustrate that practical participatory evaluation places great emphasis on evaluation as an intervention in the setting in which an evaluation is carried out. The evaluator intentionally and actively builds skills, develops knowledge, and attempts to change beliefs about evaluation in an effort to leave behind an organization that has improved capacity to evaluate and learn from evaluation.…”

“…They used online searches to create a list of publications describing each theory and, after consulting the theorists for additional writing to include in the study, analyzed five to seven texts that they believed provided the best treatments of the theorists' ideas. The students then coded the texts for 61 concepts (see Hansen, Alkin, & Wallace, 2013). Coded text was sorted into five categories mirroring the sequence of a logic model: assumptions, context, activities, consequences, external factors.…”

Towards a model for inputs evaluation for workplace HIV/AIDS IEC programme based on process evaluation theoretical framework

“…The need for a schematic presentation of an evaluation model flows from the fact that a visual depictions helps to clarify its most important features, components as well as their attributes (Hansen et al, 2012;Greene, 2013). The study's adoption of logic evaluation model is anchored on many evaluation scholars who have adopted the model to articulate, compare and develop and analyse evaluation theoretical models (Mark and Henry, 2013;Vo, 2013;Dillman, 2013;Luskin and Ho, 2013).…”

Rigour in Methods and Evaluation for Community Engagement

“…As feminist legal scholars have indicated, developing a rigorous approach to change must necessarily ask the big questions, such as whether the changes are deep or superficial changes (Bartlett, 1995). The remainder of this section, carries out an in-vestigation of the origin and interplay of concepts contributing to a rigorous theory of change, while stopping short of producing a visual representation or 'evaluation theory tree' of these ori-gins (as has been accomplished elsewhere) (Alkin and Christie, 2004;Hansen et al, 2012).…”

“…This includes contribu-tion analysis especially (Mayne, 2008;Mayne, 2012). Some are more likely to be categorized as tools rather than frameworks which carry less of a theoretical implication, such as program logic models (McCawley, circa 2000;Hansen et al, 2012) outcome mapping (Earl and Carden, 2002;Lampkin et al, 2006) and results-based management (Frechtling, 2007;Morra Imas and Rist, 2009). Some of these tools may even be reframed as 'theory of change' logic models (Lynch-Cerullo and Cooney, 2011: 370).…”