Taking stock of complexity in evaluation: A discussion of three recent publications

Gerrits, Lasse; Verweij, Stefan

doi:10.1177/1356389015605204

Cited by 17 publications

(15 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, we also draw readers' attention to applications outside the DE community of complexity-thinking in the theorization and practice of evaluating social interventions, including SIs. Authors such as Callaghan (2008), Gerrits and Verweij (2015), Mowles (2014), Rogers (2008) and Walton (2014) have been working on complexity-informed approaches in the evaluation field drawing on a range of perspectives from the complexity sciences. 3 We note, of course, that approaches other than DE and complexity-aware evaluation are being used in SI contexts.…”

Section: Social Innovation and Evaluation Practicementioning

confidence: 99%

The evaluation of social innovation: A review and integration of the current empirical knowledge base

et al. 2018

View full text Add to dashboard Cite

Social innovation has gained prominence as a way to address social problems and needs. Evaluators and social innovators are conceptualizing and implementing evaluation approaches for social innovation contexts; however, no systematic effort has yet been made to explore and assess the overlap between evaluation and social innovation based on the empirical knowledge base. We address this gap, drawing on 28 empirical studies of evaluation in social innovation contexts to describe what evaluation practices look like, what drives those practices, and how they affect social innovations. Findings indicate most had developmental purposes, emphasized collaborative approaches, and used multiple methods. Prominent drivers were a complexity perspective, a learning-oriented focus, and the need for responsiveness. Reported influences on social innovations included advancing strategies, improving delivery, balancing aggregate and local information needs, and reducing risk. Conflict resolution, the quality of relationships, and availability of time and capacity mediated these influences. More peer-reviewed empirical studies and a broader range of study designs are needed, including research on how evaluations influence social innovation processes over time, phases, space and scale.

show abstract

Section: Social Innovation and Evaluation Practicementioning

confidence: 99%

The evaluation of social innovation: A review and integration of the current empirical knowledge base

et al. 2018

View full text Add to dashboard Cite

show abstract

“…This shift in the international policy agenda warrants a move to more health systems oriented research and evaluation. Simultaneously, the latter two domains increasingly focus on “complexity” in health systems [ 2 ]. This has resulted in many different methodological responses: the defiance [ 3 , 4 ] and/or adaptation [ 5 ] of existing randomized controlled trial methodologies; the application of theory of change [ 6 ]; the rising prominence of realist evaluation [ 7 , 8 ] and, more generally, the use of theory-driven evaluations [ 9 , 10 ]; and the introduction of systems thinking [ 11 , 12 ].…”

Section: Introductionmentioning

confidence: 99%

Combining Theory-Driven Evaluation and Causal Loop Diagramming for Opening the ‘Black Box’ of an Intervention in the Health Sector: A Case of Performance-Based Financing in Western Uganda

Renmans

Holvoet

Criel

2017

IJERPH

View full text Add to dashboard Cite

Increased attention on “complexity” in health systems evaluation has resulted in many different methodological responses. Theory-driven evaluations and systems thinking are two such responses that aim for better understanding of the mechanisms underlying given outcomes. Here, we studied the implementation of a performance-based financing intervention by the Belgian Technical Cooperation in Western Uganda to illustrate a methodological strategy of combining these two approaches. We utilized a systems dynamics tool called causal loop diagramming (CLD) to generate hypotheses feeding into a theory-driven evaluation. Semi-structured interviews were conducted with 30 health workers from two districts (Kasese and Kyenjojo) and with 16 key informants. After CLD, we identified three relevant hypotheses: “success to the successful”, “growth and underinvestment”, and “supervision conundrum”. The first hypothesis leads to increasing improvements in performance, as better performance leads to more incentives, which in turn leads to better performance. The latter two hypotheses point to potential bottlenecks. Thus, the proposed methodological strategy was a useful tool for identifying hypotheses that can inform a theory-driven evaluation. The hypotheses are represented in a comprehensible way while highlighting the underlying assumptions, and are more easily falsifiable than hypotheses identified without using CLD.

show abstract

“…Moving towards 2020 we get closer to the some of the concerns of this special issue, as scholars begin to become more critical and self-reflexive about the value and use of these ideas and methods, particularly in terms of the realities of using them in the practice of evaluation (e.g. Gerrits and Verweij, 2015; Walton, 2016).…”

Section: The ‘Complexity Turn’ In Policy Evaluationmentioning

confidence: 99%

Policy evaluation for a complex world: Practical methods and reflections from the UK Centre for the Evaluation of Complexity across the Nexus

et al. 2021

View full text Add to dashboard Cite

The value of complexity science and related approaches in policy evaluation have been widely discussed over the last 20 years, not least in this journal. We are now at a crossroads; this Special Issue argues that the use of complexity science in evaluation could deepen and broaden rendering evaluations more practical and rigorous. The risk is that the drive to better evaluate policies from a complexity perspective could falter. This special issue is the culmination of 4 years’ work at this crossroads in the UK Centre for the Evaluation of Complexity Across the Nexus. It includes two papers which consider the cultural and organisational operating context for the use of complexity in evaluation and four methodological papers on developments and applications. Together, with a strong input from practitioners, these papers aim to make complexity actionable and expand the use of complexity ideas in evaluation and policy practice.

show abstract

Taking stock of complexity in evaluation: A discussion of three recent publications

Cited by 17 publications

References 34 publications

The evaluation of social innovation: A review and integration of the current empirical knowledge base

The evaluation of social innovation: A review and integration of the current empirical knowledge base

Combining Theory-Driven Evaluation and Causal Loop Diagramming for Opening the ‘Black Box’ of an Intervention in the Health Sector: A Case of Performance-Based Financing in Western Uganda

Policy evaluation for a complex world: Practical methods and reflections from the UK Centre for the Evaluation of Complexity across the Nexus

Contact Info

Product

Resources

About