Evaluation systems: How do they frame, generate and use evidence?

Abstract

How does the design of evaluation systems affect the different ways of using the results of evaluations? This article offers a conceptual model that outlines three 'ideal' types of evaluation systems. It is a heuristic tool for opening up the 'black box' of evaluation systems and assessing their qualitative differences in terms of types of 'owners' of evaluations, questions asked, methods deployed, answers provided and avenues for use of evaluative knowledge. We apply the model to study the case of the Lithuanian evaluation system. In contrast to the expectations of some of the previously developed models, it does use evaluation results, and we aim to understand why the generated evidence is more often used in some areas rather than others.

Structures and functions of complex evaluation systems: comparison of six Central and Eastern European countries

“…The issue of factors shaping structure is also open for further exploration. At this point, we may only speculate that these factors could include the formal and informal characteristics of public administration settings, cultures and traditions (Curristine et al, 2007), the nature of external pressure on national administrations (Højlund, 2014) or the power struggle between actors in the system (Martinaitis et al, 2018).…”

Grounding evaluation capacity development in systems theory

“…DEval’s expertise lies in supporting national evaluation systems (NESs). While most of the existing literature on evaluation systems is situated at the organizational or institutionalized levels (Andersen, 2020; Briceño, 2012; Leeuw and Furubo, 2008; Martinaitis et al, 2019; Raimondo, 2018), the systemic perspective adopted in this article considers all actors, processes, and structures as national evaluation system components that allow for the planning, implementation, and use of evaluations to assess national public policies. As we explain in later sections, it is important to note that the composition of a national evaluation system gradually evolves and changes (Pérez Yarahuán and Maldonado Trujillo, 2015).…”

The constitutive effects of evaluation systems: Lessons from the policymaking process of Danish Active Labour Market Policies

“…Some scholars have tried to map out the extent of the institutionalisation of evaluations in modern governments (Ahonen, 2015; Jacob et al, 2015; Martinaitis et al, 2018). These studies have provided important descriptive overviews of how governments systematise evaluation activities, but they stop short of probing the consequences of evaluation systems for the design of new policies.…”

“…Drawing on Leeuw and Furubos' conceptualisation of evaluation systems, a small empirical literature on the consequences of evaluation systems for evaluation use has emerged (Hanberger, 2011;Hanberger and Lindgren, 2018;Højlund, 2014;Lindgren et al, 2016;Martinaitis et al, 2018;Raimondo, 2018). These studies share a number of similarities with the literature on the use of performance management and monitoring in the public sector (Brodkin, 2011;Brodkin and Marston, 2013;Møller et al, 2016;Soss et al, 2011;Triantafillou, 2012).…”