Is auditing the new evaluation? Can it be? Should it be?

Abstract

Purpose The purpose of this paper is to study the changing relationship between auditing and evaluation. Over the past several years, supreme auditing institutions (SAIs) in a number of advanced democracies have evolved from conventional auditing institutions to becoming increasingly concerned with assisting policy change and administrative reform in the public sector; tasks that are traditionally associated with evaluation. The paper discusses the potential consequences of this development for the SAIs themselves as well as for the audited and reforming institutions and for policy-making. Design/methodology/approach The paper uses qualitative method and draws on the extensive literature on auditing and evaluation. The analysis has also benefitted from the authors’ recent comparative research on SAIs. Findings The findings, summarized in six points, are that the growth of auditing in areas previously assigned to evaluators, has led to a shortened time perspective; stronger emphasis on the administration of policies; increased focus on efficiency of the audited entity; greater independence from the evaluated organizations; a shift in receiver of information toward the legislature and/or the public; and improved communication. Practical implications Evaluation as a professional and scholarly field has developed theories and advanced methods to assess the effectiveness of public programs. The growth of auditing may thus change the focus and quality of policy evaluation. Originality/value The paper speaks to both scholars and practitioners. To the best of the knowledge a similar analysis has not been done before.

The ‘front stage’ of substance auditing: A study of how substance auditing is presented in performance audit reports

“…One key way of enabling this accountability relationship is to make sure there are linkages of communication between the public and their public servants (Everett, ), and PA reports are central parts of this communication (Waerness, ). Substance audits, in turn, have been regarded as key components in this communication, since such audits hold the potential to provide an answer to the question of ‘what should be’ (Everett, , p. 81) or whether the undertakings in central government ‘are working’ (Pierre, et al., , p. 730; Pollitt & Summa, , p. 2), i.e. a value‐laden answer to the question of whether or not performance is ‘good’.…”

“…This finding is interesting given some of the arguments put forward in the research community for why substance audits are important. SAIs and their PAs tend to be associated with participatory and democratic conceptions of government (Pierre et al., ), and substance auditing is understood as important for political accountability relationships. Political accountability refers to direct answerability to the general public with the aim of helping to involve people as well as mass media in policy processes and central government activities.…”

“…The facts have to be confronted with some sort of criterion or criteria before a judgement or recommendation can be arrived at’. The establishment of criteria for the PA work is perhaps the most distinguishing feature between PA and other forms of evaluation (Pierre et al., ). In order to present the results of the audit and to fulfil the assignment to establish ‘whether, at what cost, and to what degree the policies, programmes and projects of government are working’ (Pollitt & Summa, , p. 2), SAIs have to establish clear and precise criteria against which judgements about ‘good’ or ‘poor’ performance can be made.…”

“…Thus, the PA reports are expected to provide information about the auditees’ performance in a way that enhances the central government's accountability and transparency. The reports are also a way of making the audit process itself transparent, which on the one hand is necessary for the legitimacy of the audit, but on the other hand potentially exposes the audit to criticism (Pierre et al., ; Reichborn‐Kjennerud & Johnsen, ).…”

“…The reason for this criticism is that several studies of PAs (see, e.g. Bawole & Ibrahim, 2016;Glynn, Gray, & Jenkins, 1992;Grönlund, Svärdsten, & Öhman, 2011;Pierre, Peters, & de Fine Licht, 2018;van Thiel & Leeuw, 2002) have established that the SAIs tend to carry out audits of 'systems' rather than audits of 'substance' (Summa, 1999, p. 72). System audit refers to auditing the administrative systems through which the auditees' core activities are governed, whereas substance audit refers to auditing the core activity itself (see, e.g.…”