Making sense of the emerging conversation in evaluation about systems thinking and complexity science

Gates, Emily

doi:10.1016/j.evalprogplan.2016.08.004

Cited by 74 publications

(88 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Callaghan, 2008;Stame, 2004). In the last few years, as we do here, authors have reflected and reviewed the use of complexity in evaluation, for example, its use directly in evaluation scholarship (Gates, 2016;Mowles, 2014;Walton, 2014), the acceptance and use of complexity in prac-tice (Reynolds, Gates, Hummelbrunner, Marra, & Williams, 2016), and reflections on which parts of the typical evaluation process it can be most useful to (Williams, 2015). The shifting focus of these authors through time reflects the growing use and acceptability of complexity in public policy analysis and evaluation; this shift is representative of the evaluation community in the UK too.…”

Section: Introductionmentioning

confidence: 94%

“…T he growing interest in the application of insights and methodological approaches from the study of complex adaptive systems (from now on referred to simply as "complexity") in public policy in the UK is well documented (see, for example, Byrne & Callaghan, 2014;Byrne & Uprichard, 2012;Anzola, Barbrook-Johnson, & Cano, 2017). In ex-post policy evaluation in particular-which is a significant component and focus of public policy analysis in the UK-interest has grown steadily over the last 20 years (Reynolds, Forss, Hummelbrunner, Marra, & Perrin, 2012;Gates, 2016).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Reflections On the Use of Complexity-Appropriate Computational Modeling for Public Policy Evaluation in the UK

Barbrook-Johnson¹,

Schimpf²,

Castellani³

2019

JPCS

View full text Add to dashboard Cite

In the UK, calls for the application of insights from the study of complex adaptive systems to public policy evaluation are beginning to be taken seriously in government. Policymakers and analysts are accepting the fallibility of overly simplistic, definitive, or linear analysis, or are finding traditional forms of analysis and evidence less appropriate or feasible. Through our work in CECAN (the Centre for the Evaluation of Complexity Across the Nexus), we reflect on our experiences and the practical challenges of using complexity-appropriate computational modeling with policy ana

show abstract

Section: Introductionmentioning

confidence: 94%

Section: Introductionmentioning

confidence: 99%

Reflections On the Use of Complexity-Appropriate Computational Modeling for Public Policy Evaluation in the UK

Barbrook-Johnson¹,

Schimpf²,

Castellani³

2019

JPCS

View full text Add to dashboard Cite

show abstract

“…For this reason, systems thinking has been regarded as a promising option to tackle complex situations in the evaluation field. Although there have been attempts at both theory (Imam et al, 2006;Cabrera et al, 2008) and practice (Gregory, 1997;Newman et al, 2003;Burke, 2006;Fitch, 2006;Reynolds, 2006;Fredericks et al, 2008;Ulrich and Reynolds, 2010;Williams and Hummelbrunner, 2011;Martin and MacDonald, 2012;Derwisch andLowe, 2015 andHart andPaucar-Caceres, 2017) of exploring how systems thinking concepts could serve program evaluation, to date, there has not been a systematic examination of how these ideas and methods contribute to the evaluation field or a framework for understanding when and why to use them (Gates, 2016).…”

Section: A Critical Approach To Program Evaluation and Systems Thinkingmentioning

confidence: 99%

Developing a Systemic Program Evaluation Methodology: A Critical Systems Perspective

2018

View full text Add to dashboard Cite

In recent years, there has been an increased interest within the program evaluation field in the introduction of systems thinking concepts. However, most of these introductions have been primarily directed towards supporting the practice of evaluation and not towards making theoretical advancements. This article is focused on introducing systems thinking, and specifically perspectives and concepts from the work in critical systems thinking (CST), at a theoretical level in the program evaluation field, towards a reframing of Fourth Generation Evaluation methodology. The process for carrying out such a reframing is introduced, as well as a description of the major changes produced in the evaluation methodology from incorporating the CST perspective. A new model is proposed, and how this model may be beneficial for conducting an evaluation is discussed. Finally, recommendations are made for future developments. © 2018 John Wiley & Sons, Ltd.

show abstract

“…This could mean changing the way things are evaluated as the rooms are rolled out in different communities. Things like population size, underlying disease burdens, and general hospital admission numbers should be gathered as completely as possibly and used when evaluating the VHR effectiveness in different communities (Gates, 2016). The marks of improvement in an urban center will be quite different than a rural one, and the argument could be made that even if a handful of patients observe better outcomes, this is sufficient given the realities of these communities.…”

Section: Evaluation Through a Complexity Science Lensmentioning

confidence: 99%

Implementation considerations in a Complex Adaptive System

Petrie¹,

Peters²

2018

View full text Add to dashboard Cite

Executive summaryThis report is provides guidance for research teams who are currently planning or are in the midst of implementing an e-health intervention in rural communities. It describes the important factors which need to be considered when scaling-up a pilot project from one context to another, and demonstrates what a successful project needs to maximize the probability that it will achieve the desired level of spread within the healthcare system. This report can be used as a reference for people who wish to implement a novel intervention into a new environment. Ideally it will be used in the early stages of intervention design to help researchers understand how a complex adaptive system functions and why navigating one is important for the outcome of their intervention.To begin, the report covers some basic terminology used when discussing complex adaptive systems and highlights the importance of working with these ideas moving forward. Next, in-depth discussions about sense-making, leverage points, self-organization, and agent-based modelling provide evidence of the complexity of implementation. Finally, the principle of antifragility is discussed, as well as a tangible example of an intervention which has been designed with antifragility in mind. Finally, the conclusion summarizes the key findings of the report, offers future directions, and identifies some of the limitations.3 Acknowledgements

show abstract

Making sense of the emerging conversation in evaluation about systems thinking and complexity science

Cited by 74 publications

References 47 publications

Reflections On the Use of Complexity-Appropriate Computational Modeling for Public Policy Evaluation in the UK

Reflections On the Use of Complexity-Appropriate Computational Modeling for Public Policy Evaluation in the UK

Developing a Systemic Program Evaluation Methodology: A Critical Systems Perspective

Implementation considerations in a Complex Adaptive System

Contact Info

Product

Resources

About