The power and dysfunctions of evaluation systems in international organizations

Raimondo, Estelle

doi:10.1177/1356389017749068

Cited by 35 publications

(57 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This adds to the political dimensions of evaluation practice, whereby evaluation can become a tool in the defence of stakeholders' interests in political struggles for resources. 18,57 The very act of commissioning an evaluation can be a persuasive Arguably, undue emphasis on early positive achievements can preclude the evaluation of more uncertain longer-term impacts.…”

Section: E Valuati On For Prog R Amme C Apital : Per Sua S Ive Utilmentioning

confidence: 99%

“…18 Nevertheless, available research and evidence from evaluators' experiences suggest that persuasive utilisation is a common outcome of evaluation. 57,65 The prevalence of persuasive use may be linked to work-role expectations that emphasise the sharing of programme impacts with senior institutional leadership and programme stakeholders. 65 Correspondingly, it may be a valid by-product of evaluation in some contexts, when it is merited by evaluation findings.…”

Section: E Valuati On For Prog R Amme C Apital : Per Sua S Ive Utilmentioning

confidence: 99%

“…ONYURAcritical part of evaluation practice, problems arise when the evaluation attends exclusively to a narrow set of outcomes without seeking to question programme implementation processes, underlying assumptions or unintended outcomes. Further, seeking impact for symbolic purposes can lead to undue focus on outcomes that can be achieved and measured within shorter-term reporting cycles at the expense of longer-term outcomes 57. Thus, although such narrowly…”

mentioning

confidence: 99%

See 2 more Smart Citations

Useful to whom? Evaluation utilisation theory and boundaries for programme evaluation scope

Onyura

2020

Medical Education

View full text Add to dashboard Cite

Context: Programme evaluation is perpetually mandated in health professions education. Correspondingly, there has been an expansion in prescriptive methodological guides about 'how' to engage in various best practices in evaluation. However, what has gained less attention is an examination of the 'value' that different stakeholders seek to gain from programme evaluation. Evaluation utilisation theory and research can help us understand the diversity in both the driving forces for and the impact of programme evaluation. Awareness of the heterogeneity of evaluation utilisation priorities has implications for evaluation practices, including both methodological choices and understanding of the impact of programme evaluation in our field. Methods: In this article, I expound on the concept of evaluation utilisation by drawing on evaluation theory and research. 'Evaluation utilisation' refers to the application of programme evaluation processes and findings to influence thinking and action. Herein, different forms of 'evaluation utilisation' (including instrumental, conceptual, process and persuasive (symbolic and legitimative) utilisation) are discussed, as well as the related concept of 'evaluation misuse'. Furthermore, how the prioritisation of different forms of 'evaluation utilisation' can influence the scope and impact of evaluation scholarship are also discussed. Conclusions: Programme evaluation is a form of inquiry that requires more than the exercise of robust methodological techniques. Rather, it necessitates attention to the, sometimes divergent, priorities of different stakeholder groups. Although there is scant research on evaluation practices in health professions education, evaluation utilisation theory can inform critical examination of evaluation practices and impact in our field. Critically, understanding this body of work can help inform those engaged in evaluation about what they are (or should be) prioritising when they conduct programme evaluation, and better align evaluation methodologies with their scholarly, curricular and administrative intentions. Implications for future research and high-quality, transparent evaluation scholarship are presented. How to cite this article: Onyura B. Useful to whom? Evaluation utilisation theory and boundaries for programme evaluation scope.

show abstract

Section: E Valuati On For Prog R Amme C Apital : Per Sua S Ive Utilmentioning

confidence: 99%

Section: E Valuati On For Prog R Amme C Apital : Per Sua S Ive Utilmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Useful to whom? Evaluation utilisation theory and boundaries for programme evaluation scope

Onyura

2020

Medical Education

View full text Add to dashboard Cite

show abstract

“…Several evaluation authors, including Raimondo (2018), conceive of "evaluation systems" as different from what political scientists would perhaps normally speak of as "polities" or "systems of governance." They distinguish between the system of performance monitoring; the systems of performance audit, inspection, and oversight; the system of (quasi)experimental evaluations and the evidence-based policy movement; the accreditation and evaluation system; and the monitoring and evaluation system.…”

Section: Evaluation Institutions (Political Science Perspective)mentioning

confidence: 99%

The Politicisation of Evaluation: Constructing and Contesting EU Policy Performance

Stephenson

Schoenefeld²,

Leeuw

2019

Polit Vierteljahresschr

View full text Add to dashboard Cite

Although systematic policy evaluation has been conducted for decades and has been growing strongly within the European Union (EU) institutions and in the member states, it remains largely underexplored in political science literatures. Extant work in political science and public policy typically focuses on elements such as agenda setting, policy shaping, decision making, or implementation rather than evaluation. Although individual pieces of research on evaluation in the EU have started to emerge, most often regarding policy "effectiveness" (one criterion among many in evaluation), a more structured approach is currently missing. This special issue aims to address this gap in political science by focusing on four key focal points: evaluation institutions (including rules and cultures), evaluation actors and interests (including competencies, power, roles and tasks), evaluation design (including research methods and theories, and their impact on policy design and legislation), and finally, evaluation purpose and use (including the relationships between discourse and scientific evidence, political attitudes and strategic use). The special issue considers how each of these elements contributes to an evolving governance system in the EU, where evaluation is playing an increasingly important role in decision making.Zusammenfassung Obwohl systematische Policy-Evaluation bereits seit vielen Jahrzehnten betrieben wird und insbesondere innerhalb der Institutionen der Europäischen Union (EU) und der EU-Mitgliedstaaten stark gewachsen ist, wird sie weiterhin größtenteils von der politikwissenschaftlichen Literatur vernachlässigt. Bestehende Arbeiten in der Politikwissenschaft und der Policy-Analyse konzentrieren sich meist auf andere Themen, wie Agenda-Setting, Policy-Entstehung, politische Entscheidungsprozesse oder die Implementierung öffentlicher Politiken. Während an einigen Stellen vereinzelte Forschungsarbeiten zur Evaluation in der EU erscheinen, die sich meist auf Policy-Effektivität konzentrieren (ein Kriterium unter vielen), fehlt bisher ein strukturierter Ansatz. Das aktuelle PVS-Schwerpunktheft stößt in diese Forschungslücke und konzentriert sich auf vier Schlüsselaspekte: Evaluationsinstitutionen (inklusive ihrer Regeln und Kulturen), Evaluationsakteure (Kompetenzen, Macht, Aufgaben, Interessen und Interaktionen), Evaluationsmethoden und -theorien (inklusive Folgenabschätzung und deren Einfluss auf das Policy-Design und Gesetzgebung) und zum Schluss Evaluationszwecke und -nutzen (inklusive der Beziehung zwischen Diskurs und wissenschaftlicher Evidenz, politische Einstellungen und strategischem Nutzen). Das Schwerpunktheft betrachtet, inwiefern diese Elemente zur Entwicklung des EU-Governance-Systems beitragen, in dem Evaluation eine immer wichtigere Rolle in Entscheidungsprozessen spielt. Schlüsselwörter EU-Institutionen · Performanz · Evalationsdebatten · Lernen · Evaluationsverwendung 2 Bridging Political Science and Evaluation Studies: Institutions, Actors, Design, and UseSo what is the existing divide acros...

show abstract

“…Existing research highlights that powerful stakeholders can use the institutional context surrounding evaluations to exercise their influence (Azzam 2010;Højlund 2014;Raimondo 2018). In IOs, this could be done through the control over IO evaluation systems resources, i.e.…”

Section: Introductionmentioning

confidence: 99%

Explaining the political use of evaluation in international organizations

Eckhard

Jankauskas

2020

Policy Sci

View full text Add to dashboard Cite

Despite a growing literature on the politics of evaluation in international organizations (IOs) and beyond, little is known about whether political or administrative stakeholders indeed realize ex-ante political interests through evaluations. This is, however, especially important considering the booming business of evaluation and the proliferation of institutional assessments both in domestic and international politics. We argue that formally independent IO evaluation units informally orientate towards either member states or the IO administration, depending on who controls the unit’s budget, staff, and agenda resources. This should enable either actor to also use evaluation results along pre-defined strategic interests. Interview data gathered among evaluators, secretariat officials, and member state representatives of six IOs support the expected pattern, highlighting striking differences in the orientation of evaluation staff and evaluation use. Findings challenge the technocratic, apolitical image of evaluation, offering practical and theoretical implications for future research.

show abstract

The power and dysfunctions of evaluation systems in international organizations

Cited by 35 publications

References 55 publications

Useful to whom? Evaluation utilisation theory and boundaries for programme evaluation scope

Useful to whom? Evaluation utilisation theory and boundaries for programme evaluation scope

The Politicisation of Evaluation: Constructing and Contesting EU Policy Performance

Explaining the political use of evaluation in international organizations

Contact Info

Product

Resources

About