Towards systemic evaluation in turbulent times – Second-order practice shift

Abstract

The need for, and possibilities of, a second-order shift in evaluation practice are explored. Second-order evaluation practice enables an evaluator to improve practice as a skilled practitioner, acknowledging her embeddedness within an evaluand. The article explores evaluation practice as experienced by professional evaluators, using ideas from developmental evaluation coupled with systemic evaluation in the tradition of systems thinking in practice. Systemic evaluation aims to capture systemic sensibilities – the bigger picture – of complex turbulent situations of change underpinning evaluands. Attributes of second-order practice with systemic evaluation are understood as being aligned with both systemic and systematic modes of evaluation praxis. Personal experiences are provided where this juxtaposing praxis has been found wanting. By example, a systems thinking in practice framework is explored as heuristic support for making a second-order practice shift. The article concludes with a discussion of some implications for developments in professionalising evaluation practice and research.

From Understanding to Impactful Action: Systems Thinking for Systems Change in Chronic Disease Prevention Research

“…Articulating a systems theory of change is an important part of any ongoing social learning process focused on creating change [13,21] and has significant potential to guide the incorporation of a systemic lens in prevention research, to guide future research and action. Within public health, intervention and evaluation studies, and translation research have assisted to shift the field beyond understanding and description in the direction of change through the evidence-based movement [51,52].…”

“…Therefore, when engaging in research designed to create change, processes of reflection, action, and feedback must be ongoing [13,20]. Further, engaging in reflective practice is a hallmark of becoming a skillful practitioner and must include both selfreflection and reflection on the research project itself [21]. The formulated PSCF has potential as a tool for retrospective review of existing work, and as a precursor for planning more impactful research.…”

“…We thus contend that undergoing the type of reflective process enabled through the PSCF is valuable to do both retrospectively and as part of a research intervention planning process (Figure 1). Ideally, such cycles become a continuous process whereby reflection and planning become synonymous and feed into further research [20,21]. The development of the PSCF from the ACF was based on a single retrospective study.…”

“…Foster-Fishman and Watson's ABLe change framework (ACF) [13] was selected and adapted for the Prevention Systems Change Framework (PSCF) and piloted to develop a theory of systems change for prevention. This iterative process of adapting and exploring was aligned with systemic evaluation practice characterised by in-depth and ongoing empirically-based reflection [20,21].…”

Embedded Researchers as Part of a Whole Systems Approach to Physical Activity: Reflections and Recommendations

“…It requires an intricate and nuanced understanding of local processes and what is needed to secure long-term and sustainable change. Interest around systems thinking in evaluation has gained traction over the last few decades [3,4]. There are a number of approaches which can be adopted to capture the complex and turbulent nature of WSAs [3].…”