A Systematic Review of Theory-Driven Evaluation Practice From 1990 to 2009

Abstract

Although the general conceptual basis appeared far earlier, theory-driven evaluation came to prominence only a few decades ago with the appearance of Chen’s 1990 book Theory-Driven Evaluations. Since that time, the approach has attracted many supporters as well as detractors. In this paper, 45 cases of theory-driven evaluations, published over a twenty-year period, are systematically examined to ascertain how closely theory-driven evaluation practices comport with the key tenants of theory-driven evaluation as described and prescribed by prominent theoretical writers. Evidence derived from this review to repudiate or substantiate many of the claims put forth both by critics of and advocates for theory-driven forms of evaluation are presented and an agenda for future research on the approach is recommended.

"Development of the Indicators and Instrument to Evaluate the Effectiveness of the Professional Nurse Training Project: An Application of Kirkpatrick’s Model and Theory-Based Evaluation Approaches "

“…According to Chen (1990), The theory-based evaluation approach can be explained what factors were responsible for the program's success or failure (Birckmayer & Weiss, 2000;Chen, 1990;Chen, 2005;Coryn et al, 2011). The development of the indicators and instrument to evaluate the effectiveness of the professional nurse training project is an important process in the theory-based evaluation approach.…”

A theory-based evaluation of a community-based funding scheme in a disadvantaged suburban city area

“…Thus, program theory can help to build a holistic understanding of social interventions (Chen, 2005). Program theory can also be used to guide evaluation (Coryn et al, 2011). That is, by making explicit the 'theory' or assumptions behind an intervention, program theory can serve to identify evaluation questions which can be examined, thereby enabling assessment of how well the theory 'stands up' to scrutiny (Donaldson, 2007).…”

“…That is, the execution of a theory-based evaluation can help to provide a useful schematic overview and important insights into what these kinds of community-based funding schemes are trying to achieve and how their objectives and interim achievements aim to contribute to positive longer-term outcomes. Thus, this kind of evaluation can help program designers and implementers, as well as researchers develop evaluative questions which can be tested at a later stage to examine whether the scheme has met its intended goals (Coryn et al, 2011).…”

Developing a Research Agenda for Impact Evaluation in Development

“…For example, Coryn et al(2011) claimed to have conducted a systematic review of theory-driven evaluation, but, for reasons not explained or justified, restricted the search to 'traditional, mainstream scholarly outlets including journals and books', 'excluding other sources including doctoral dissertations, technical reports, background papers, white papers, and conference presentations and proceedings ' (p. 208). Journal articles represent a small and unrepresentative sample of evaluation practice, as illustrated in Figure 1.…”