Toward Valuing With Critical Systems Heuristics

Abstract

Evaluation is defined by its central task of valuing—the process and product of judging the merit, worth, or significance of a policy or program. However, there are no clear-cut ways to consider values and render value judgments in evaluation practice. There remains contention in the evaluation field about whether and how to make value judgments. No approach to valuing eliminates the uncertainty, plurality, and potential for conflict that comes with considering values. This article explores what critical systems heuristics (CSH), an area of applied systems thinking, might contribute to four long-standing issues regarding valuing: envisioning the social value of evaluation, framing the evaluand and evaluation, selecting and justifying criteria, and determining the roles of the evaluator(s) and stakeholders in valuing. CSH contributes concepts and tools that, in theory, support more reflective, responsible valuing although further practical application is needed.

Critical Systems Heuristics: The Idea and Practice of Boundary Critique

“…The basic theme of this concluding section is that boundary reflection and discourse have a lot to do with who we are and how we work as professionals, that is, with our sense of professional identity and competence (Ulrich 2000(Ulrich , 2001. Boundary critique with CSH has for example been used to explore attributes of evaluation as an emerging profession (Schwandt 2015;Reynolds and Schwandt 2017;Gates 2018) and to reflect on the personal ethical decisions involved (Levin-Rozalis 2014, Ch. 9, and 2015).…”

The overall evaluative conclusion as an element of evaluation practice

“…In many evaluations the lack of any explicit justification of the synthesis procedure is the Achilles heel of the whole effort … the last step is simply a reaction to the rest, a high-inference judgment which is rather obviously idiosyncratic. (1994, p. 367) An alternative perspective: A holistic approach An experiential approach contrasts with the more widespread criterial approach to synthesis (Gates, 2017). This approach is most evident in the work of Stake, particularly his responsive evaluation (Stake, 2004).…”

“…The contrast between this holistic, experiential approach with the more common criterial or analytic approach is most evident in Scriven's (1994) paper entitled "The Final Synthesis", presenting the criterial approach, and Stake's response to it, written with a group of his graduate students (Stake et al, 1997). These papers are cited in other literature as presenting two important perspectives on valuing, and particularly on the drawing of overall evaluative conclusions (Gates, 2017;House & Howe, 1999;Julnes, 2012aJulnes, , 2012b. Stake et al (1997) don't reject the view that evaluation is about judging the merit or worth of an evaluand, or that this culminates in a synthesis or overall evaluative conclusion.…”

“…Absent from the literature is research that explores client or stakeholder views on the inclusion or exclusion of an overall conclusion or synthesis, and how this influences their engagement with and use of evaluation findings. Gates (2017) concludes that the making of value judgements is still one of the least clear-cut aspects of evaluation practice.…”

“…The questions of who values, how, and under what conditions are more explicitly considered by some theorists than others, and are contentious aspects of evaluation practice (Alkin, Vo, & Christie, 2012;Gates, 2017). This literature review explores one aspect of valuing-the valuing involved in reaching a warranted conclusion about the overall merit, worth, or significance of an evaluand.…”