No abstract

Selecting and justifying relevant criteria is critical to defensible evaluative reasoning, yet there is little advice in the literature to guide practitioners in criteria development. In this article, we introduce a framework that draws on normative ethical perspectives to systematically identify and justify relevant dimensions of value for a public sector–funded program. We illustrate, through an example taken from a recent research project, how evaluators might use the framework to be more thoughtful about identifying and selecting the values against which judgements are made. The potential strengths and limitations of the framework are then discussed.

Practitioner competence is a critical ingredient in the development of a robust, valid and equitable evaluation. In Aotearoa New Zealand the evaluator competencies identify cultural competence as a core capability. There are some particular challenges that Pākehā (New Zealand European) evaluators face in developing this competency. In grappling with these complex challenges, and in the absence of a pragmatic and systematic way of responding, the writers discuss the use of a heuristic they have developed that may aid enquiry and support evaluators to work in a culturally responsive manner. Three case examples are presented for applying the heuristic in practice. The benefits of, and insights from, using the heuristic are discussed. 1 The term 'Pākehā' is contested. For the purpose of this article we use it to refer to New Zealanders of European descent, however people self-identify. Issues of identity are discussed more fully in an earlier article by the authors (Torrie,

A new intervention that is designed to be adaptive presents a challenge for evaluators when developing a logic model at the start of a government policy evaluation. Our task became more problematic when the early data collection identified that participant groups had differing ideas about what the policy was intended to achieve. Participants also tended to ‘deconstruct’ the policy and focus only on the parts for which they were responsible. Realising that traditional logic modelling (that is, a linear approach to describing policy inputs and outcomes developed at the beginning of the evaluation) would provide a simplified picture of the policy, we decided to ground the logic in the evidence. This approach provided a more in-depth understanding of how the policy components interact with each other. Our logic became a vehicle to reconceptualise the policy through presenting an alternative perspective to understanding and communicating how the policy works on the ground.

Crises may present unprecedented challenges that require people to think outside their traditional boxes. During COVID-19, many of us have seen officials and experts come together to share information and simultaneously respond to an emerging issue. For an evaluator working at the coalface of the pandemic response it can be an opportunity to draw from their kete (basket) of evaluation tools and matrixes to support the decision-making process to be as defensible as possible. This praxis article describes the process of three evaluators reflecting and discussing one such scenario and exploring how an evaluation-specific tool based on normative ethical theories could provide some benefit.