The institutionalization of evaluation matters: Updating the International Atlas of Evaluation 10 years later

Abstract

This text provides a comparative cross-country analysis of evaluation culture and the institutionalization of evaluation. The countries included in this research are the 19 OECD countries examined by the authors of the International Atlas of Evaluation 10 years ago (Furubo et al., 2002). The analysis is based on the results of an expert survey of four to five evaluation experts from different backgrounds for each country, as well as additional information from the literature. Using the nine indicators from Furubo et al. (2002) with a focus on the institutional characteristics of reforms, trends in evaluation culture over the last decade have been identified.

Governmentalities without policy capacity

“…This industry has a strong interest in promoting as universally recognized and endorsed a set of standards and indicators with which to measure the value of government decisions. It is entirely in the interest of the governments subject to evaluation to align themselves formally with these criteria and to have their actual compliance overestimated, even when a bottom-up verification contradicts such alignment (Jacob et al 2015). The win-win outcome of this game between evaluator and evaluated requires that the dissimilar logics be regarded as mere anomalies, as temporary deficiencies easy to remedy, perhaps by increasing the number of experts and consultants.…”

Governing policy evaluation? Towards a new typology

“…Evaluation scholars have implicitly raised these questions before. Writing in the pages of this journal, Jacob et al (2015) considered pluralistic evaluation systems to be more advanced than monocentric or hierarchical ones, but did not fully justify and evidence their claims. Their attempt was symptomatic of a collective failure to draw on governance theories to comprehend evaluation activities, which often bring together multiple actors and interests.…”

“…Stern (2009) has estimated that the European institutions spend approximately 45 million Euros per year on evaluation; Hojlund (2015) has calculated that the European Commission alone employs 140 staff to manage it. However, government-driven climate policy evaluation activities remain highly differentiated across the EU, where countries such as Germany and the UK have significant evaluation capacities, but southern and new member states exhibit much lower activity levels (AEA et al, 2009: 33;Jacob et al, 2015). Outside government, there are also many other actors, such as environmental groups, that should be accounted for (e.g.…”

“…EEA, 2016), coupled with a corresponding growth in investments in evaluation, growing evaluation communities, evaluation activities, and outputs (e.g. Furubo et al, 2002;Jacob et al, 2015;Mastenbroek et al, 2015;Toulemonde, 2000), now seems an opportune moment to pose important questions about the governance of evaluation itself (see, e.g. Hanberger, 2012;Stame, 2006).…”

“…Haug et al, 2010;Hildén et al, 2014;Huitema et al, 2011;Mickwitz, 2013). Growing attention to evaluation and a wide variety of actors, practices, and evaluation outputs make climate policy a suitable area in which to explore the everyday practices and governance of evaluation within a single political system, namely the EU (see also Jacob et al, 2015;Stern, 2009).…”