Towards Systemic Evaluation

Abstract

Problems of conventional evaluation models can be understood as an impoverished 'conversation' between realities (of non-linearity, indeterminate attributes, and everchanging context), and models of evaluating such realities. Meanwhile, ideas of systems thinking and complexity science -grouped here under the acronym STCSstruggle to gain currency in the big 'E' world of institutionalised evaluation. Four evaluation practitioners familiar with evaluation tools associated with STCS offer perspectives on issues regarding mainstream uptake of STCS in the big 'E' world. The perspectives collectively suggest three features of practicing systemic evaluation: (i) developing value in conversing between bounded values (evaluations) and unbounded reality (evaluand), with humility; (ii) developing response-ability with evaluand stakeholders based on reflexivity, with empathy; and (iii) developing adaptive rather than mere contingent use(fulness) of STCS 'tools' as part of evaluation praxis, with inevitable fallibility and an orientation towards bricolage (adaptive use). The features hint towards systemic evaluation as core to a reconfigured notion of developmental evaluation.

Participatory systems mapping for complex energy policy evaluation

“…Qualitative Comparative Analysis, Befani, 2013; Byrne, 2013; and causal loop diagrams and system dynamics, Dyehouse et al, 2009; Fredericks et al, 2008; Grove, 2015), to reflections on the success and value of complexity’s application (e.g. Gates, 2016; Mowles, 2014; Reynolds et al, 2016; Walton, 2014; Williams, 2015).…”

Reflections On the Use of Complexity-Appropriate Computational Modeling for Public Policy Evaluation in the UK

“…Callaghan, 2008;Stame, 2004). In the last few years, as we do here, authors have reflected and reviewed the use of complexity in evaluation, for example, its use directly in evaluation scholarship (Gates, 2016;Mowles, 2014;Walton, 2014), the acceptance and use of complexity in prac-tice (Reynolds, Gates, Hummelbrunner, Marra, & Williams, 2016), and reflections on which parts of the typical evaluation process it can be most useful to (Williams, 2015). The shifting focus of these authors through time reflects the growing use and acceptability of complexity in public policy analysis and evaluation; this shift is representative of the evaluation community in the UK too.…”

Ecological Justice for Nature in Critical Systems Thinking

“…Method selection can be transdisciplinary and partitioned to adapt the tools of inquiry with inventiveness and creativity in a process Reynolds et al . () refer to as the ‘art of bricolage’ (p. 671). This then is not a question of what method(s) but of the assumptions we use with method deployment as methods are in use for particular purposes and contain assumptions around their use.…”