How to Manage Pressure to Change Reports: Should Evaluators Be Above Criticism?

Abstract

When is it appropriate, and consistent with ethical standards, for evaluators to modify reports in response to “requests” from clients to do so? Some evaluators may view any pressure to make any changes in their reports as an attack on their independence. While, clearly, it is not acceptable for evaluators to be asked to distort or misrepresent findings and conclusions, evaluators, no less than anyone else, should be open to feedback and not be above criticism. This article discusses under what circumstances and in what forms it may be appropriate, or not, for evaluators to be given feedback and requested to consider appropriate modifications to their reports. It then identifies steps that evaluators can take to minimize inappropriate “suggestions” for change, and what they can do in the event that this does happen. It also considers implications, and strategies, for internal as well as for external evaluators.

Navigating the awkward, challenging social context of evaluation

“…There has been development and then revision of taxonomies, standards and competency frameworks, which include nuanced interpersonal skills (American Evaluation Association [AEA], 2018; ANZEA, 2011; Australian Evaluation Society [AES], 2013; Stevahn et al, 2005). The quality of the evaluation process and findings have been strongly linked to the quality of interactions and engagement (Abma, 2006; King & Stevahn, 2013; Perrin, 2019; Taut & Alkin, 2003).…”

“…Literature today acknowledges that interpersonal, social and cultural factors are embedded in the art and science of evaluation, and critical to effective practice: ‘Evaluations where evaluators hide behind their data and fail to engage with stakeholders are not likely to have much credibility or to lead to use’ (Perrin, 2019, p. 373). Research that closely examines social competencies is not commonplace (Abma, 2006), though practical guides that include evaluation-specific interaction techniques are emerging (e.g.…”

“…Although most jobs require some rapport-building, in evaluation a high skill level is required. ‘Arguably, the most important factor related to the success, acceptability, and ultimate use of an evaluation is how well an evaluator engages with potential users’ (Perrin, 2019, p. 360).…”

“…A large part of gaining stakeholders’ trust is the establishment of professional credibility (Becker et al, 2015; Grob, 2014; Patton, 2012; Perrin, 2019; Skolits et al, 2009; Taut & Alkin, 2003). All while employing comforting and rapport-building strategies, evaluators also need to project expertise, fair-handedness and professionalism in their interactions.…”

“…Recognising these dynamics can also usefully inform evaluators’ on-the-ground social interactions. Perrin (2019), for example, recalls having to quickly redirect an unwelcome manager who attempted to join a staff focus group. In each project, evaluators work in new circumstances with new assortments of groups and individuals, ‘each with their own values, agendas, motives, and behaviours’ (Morabito, 2002, p. 327).…”