Using Actor-Based Theories of Change to Conduct Robust Evaluation in Complex Settings

Abstract

The use of theories of change (ToCs) is a hallmark of sound evaluation practice. As interventions have become more complex, the development of ToCs that adequately unpack this complexity has become more challenging. Equally important is the development of evaluable ToCs, necessary for conducting robust theory-based evaluation approaches such as contribution analysis (CA). This article explores one approach to tackling these challenges through the use of nested actor-based ToCs using the case of an impact evaluation of a complex police-reform program in the Democratic Republic of Congo, describing how evaluable nested actor-based ToCs were built to structure the evaluation.

Using contribution analysis to evaluate large-scale, transformation change processes

“…This can be practically difficult to achieve, due to actors' limited availability, limited financial resources or a much shorter timeframe for the evaluation. We would therefore agree with Koleros and Mayne (2019), who highlight the significant time commitment required to develop iterative impact stories, and both the importance -and challenge -of stakeholder participation in that process. For example, a 'high level' expected impact for the project envisaged a 20 per cent reduction in the carbon footprint of enterprises working within the ecosystem.…”

Brokering Community-Based Research: Evaluating the Impacts of U-Links Centre for Community-Based Research on a Rural Canadian County

“…Contribution analysis and realist evaluation can coexist as layered approaches within one program evaluation, with the former providing broad strategic learning about implementation and the latter focusing on smaller and more promising contexts required in the embedded program theory (Blamey & Mackenzie, 2007;Koleros & Mayne, 2019).…”

“…The research builds on Jagosh et al 2015, who first looked at a realist evaluation of community-based research, and on modern evaluations theories such as nested actor-based theories of change (Koleros & Mayne, 2019) and linked contribution and realist approaches (Whynot, Lemire, & Montague, 2018), exemplifying the importance of both relationships and resources in enabling community-based research mobilization and supporting the findings of Balazs and Morello-Frosch (2013). We feel that evaluations approached from a participatory mindset and motivated by organizational growth rather than funder-imposed accountability measures can address many of these challenges.…”

“…illustrates U-Links' theory of change, revealing their logic and assumptions regarding community-based research and impact in Haliburton County and providing the baseline indicators necessary for the evaluation. U-Links' theory of change was developed from the workshop and draws inspiration fromMorton (2015) andKoleros and Mayne (2019).…”

Participatory action research to pilot a model of mental health service user involvement in an Ethiopian rural primary healthcare setting: study protocol

“…Second, although many programme theories articulate intended changes [98], engage with the complexity of interventions and provide a framework to guide action, monitoring and evaluation [99], there is little empirical evidence of how theories can be applied in practice [100][101][102]. Hence, there have been calls for PAR in theory-based implementation and evaluations [103][104][105]. Early integration of Theory of Change (ToC) and PAR during planning and implementation is recommended [103,106,107].…”