Assuring the quality of evaluative information: theory and practice

Schwartz, Robert; Mayne, John

doi:10.1016/j.evalprogplan.2004.10.001

Cited by 27 publications

(17 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The quality of policy monitoring data also relates to its relevance for the policy at hand. 5 What quality is (and for whom) will likely remain contested because it is intimately linked with various potential purposes of policy monitoring and evaluation, including accountability and learning functions, as well as politics (Schoenefeld and Jordan 2019;Schwartz and Mayne 2005). A policy monitoring system may originally be designed for checking policy goal attainment, but once the system starts producing information, this information may be used in many different ways.…”

Section: Qualitymentioning

confidence: 99%

Policy Monitoring in the EU: The Impact of Institutions, Implementation, and Quality

Schoenefeld¹,

Schulze²,

Hildén³

et al. 2019

Polit Vierteljahresschr

View full text Add to dashboard Cite

Policy monitoring is often seen as a crucial ingredient of policy evaluation, but theoretically informed empirical analyses of real-world policy monitoring practices are still rare. This paper addresses this gap by focusing on climate policy monitoring in the European Union, which has a relatively stringent system of greenhouse gas monitoring but a much less demanding approach to monitoring policies. It explores how institutional settings, policy implementation, and the quality of information may impact the practices and politics of policy monitoring. Drawing on quantitative regression models and qualitative interviews, it demonstrates that policy monitoring has evolved over time and is itself subject to implementation pressures, but also exhibits learning effects that improve its quality. In further developing both everyday policy monitoring practices and academic understanding of them, there is a need to pay attention to their design-specifically, the impact of any overarching rules, the institutional support for implementation, and the criteria governing the quality of the information they deliver. In short, policy monitoring should be treated as a governance activity in its own right, raising many different design challenges.

show abstract

Section: Qualitymentioning

confidence: 99%

Policy Monitoring in the EU: The Impact of Institutions, Implementation, and Quality

Schoenefeld¹,

Schulze²,

Hildén³

et al. 2019

Polit Vierteljahresschr

View full text Add to dashboard Cite

show abstract

“…La qualité d'une évaluation constitue l'indicateur qui a l'association statistique la plus forte avec l'utilisation des résultats (Huie Hofstetter et Alkin, 2003;Cousins et Leithwood, 1986). Aussi, la qualité d'une évaluation en général, et sa qualité méthodologique en particulier, augmenterait les probabilités que les résultats évaluatifs aient une influence sur les processus décisionnels en ce qui a trait à la vie d'un programme (Schwartz et Mayne, 2005a). En effet, la rigueur de la démarche évaluative et la crédibilité des résultats contribuent à la prise de décisions informées et donc davantage rationnelles (Daigneault, 2008).…”

Section: Méta-évaluation Et Analyse Critique Centrée Sur Le Devis D'éunclassified

Un outil à visée pédagogique pour discuter de méthodologie

Mediell

Dionne

2018

Canadian Journal of Program Evaluation

View full text Add to dashboard Cite

Dans le cadre de cet article, nous abordons l'importance de la communication de la démarche d' évaluation (processus, méthodologie, résultats et limites de l' évaluation), et ce, dans le but de favoriser l'utilisation des résultats et la mise en oeuvre des recommandations. Nous proposons un outil méta-évaluatif formatif à visée principalement pédagogique, centré sur les aspects méthodologiques du processus évaluatif afin d' accompagner les évaluateurs, et plus particulièrement les évaluateurs novices, dans la planification, la mise en oeuvre et la communication rigoureuses de la méthodologie. Nous portons particulièrement notre attention sur la communication de la démarche d' évaluation de programme par l'intermédiaire de rapports d' évaluation (techniques et finaux) qui sont généralement le véhicule offrant le maximum d'informations concernant à la fois les résultats et le processus évaluatif. Bien que nous soyons conscients que d' autres moyens de communication existent (ex. : articles publiés dans des revues scientifiques), leur format ne permet pas toujours de fournir l' ensemble des informations inhérentes autant aux résultats qu' à la démarche mise de l' avant par le(s) évaluateur(s) puisqu' on y présente généralement les faits saillants sans s' attarder aux aspects méthodologiques.Puisque notre propos touche spécifiquement aux considérations méthodologiques, il nous apparaissait plus pertinent de nous appuyer sur les informations contenues dans les rapports. Ce choix n' est pas anodin, puisque la qualité d'une évaluation repose, entre autres, sur une méthodologie avérée permettant de fournir les données probantes nécessaires à l' exercice du jugement. Aussi, nous discuterons du rôle de la méta-évaluation (MEV) dans l' enrichissement de la pratique des évaluateurs et la promotion de la qualité d'une évaluation de programme; et nous présenterons un outil méta-évaluatif que nous avons conçu afin de favoriser l' opérationnalisation de méthodologies évaluatives de qualité et la mise en oeuvre de pratiques évaluatives efficaces. Abstracts:In this article, we discuss the importance of communicating the evaluation approach (process, methodology, results, and limits) to promote the use of results and the implementation of recommendations. We present an education-focused, meta-evaluative training tool based on the methodological aspects of the evaluation process, and designed to support evaluators, particularly novice evaluators, in the rigorous planning, implementation and communication of methodology. We are Auteur-ressource : Maud Mediell, mmedi042@uottawa.ca © 2018 Canadian Journal of Program Evaluation / La Revue canadienne d'évaluation de programme 33.1 (Spring / printemps), 90-113focusing on communicating the program evaluation approach though evaluation reports (technical and final) that are usually the means offering the most information on both results and the evaluation process. We realize that there are other means of communication (i.e., journal-published articles), but their format doesn't always allo...

show abstract

“…Our findings can be instructive for other second wave countries, in which evaluation practices are also recently introduced. Besides, our analysis offers an alternative to the studies on evaluation quality assurance that exclusively focus on the world leaders in the practice of evaluation (e.g., Schwartz & Mayne, 2005).…”

Section: Policy Evaluation In Flandersmentioning

confidence: 99%

“…Systematic insights into the conditions of the usage of evaluation quality arrangements can moreover be useful for practice that seek professional development, and advocate better evaluation policies (Cooksy & Caracelli, 2005;Cooksy & Mark, 2012). The quality with which evaluations are conducted is after all considered to be one of the determinants of their use (Cooksy & Caracelli, 2005;European Commission, 2013;Leviton & Hughes, 1981;Schwartz & Mayne, 2005). In the following section, we explain how we conceive the outcome variable of this research, i.e., the extent of evaluation quality assurance by public sector organizations.…”

mentioning

confidence: 99%

Explaining Organizational Variety in Evaluation Quality Assurance: Which Conditions Matter?

Pattyn

Brans

2014

International Journal of Public Administration

View full text Add to dashboard Cite

The growth in evaluation practice has not always gone hand in hand with an increase in the actual quality of these evaluations. This article addresses this concern, by examining the factors that explain organizational variety in the application of evaluation quality assurance measures. We present the results of an analysis of 18 Flemish (Belgian) public sector organizations. To unravel different explanatory trajectories, we combine two comparative methods that rely on Boolean analysis: the most similar different outcome/most different similar outcome technique on the one hand, and crisp set qualitative comparative analysis on the other hand.

show abstract

Assuring the quality of evaluative information: theory and practice

Cited by 27 publications

References 16 publications

Policy Monitoring in the EU: The Impact of Institutions, Implementation, and Quality

Policy Monitoring in the EU: The Impact of Institutions, Implementation, and Quality

Un outil à visée pédagogique pour discuter de méthodologie

Explaining Organizational Variety in Evaluation Quality Assurance: Which Conditions Matter?

Contact Info

Product

Resources

About