No abstract

An important motivation for the institutionalization of evaluation culture in countries around the world is the belief that accountability and transparency will thereby be enhanced. We subject this narrative about evaluation’s contribution to good governance to empirical analysis. We also argue that the meaning and relevance of this general narrative differs across national contexts. We build on data from a systematic assessment of evaluation culture in 19 countries (Jacob et al., 2015), add one country using the same systematic approach, and combine these findings with an indicator of transparency in government provided by Transparency International. We find a positive correlation between evaluation culture and transparency, and discuss threats to a causal interpretation hereof. We go into depth with two particular countries at either end of the transparency scale. We argue that the meaning of the link between evaluation culture and transparency differs whether the chosen perspective is generally comparative or situated in particular national, historical, and political contexts. In countries where transparency is high already, there might be diminishing marginal returns on evaluation, at least regarding its contribution to accountability and transparency.

No abstract

The United Kingdom Research and Innovation (UKRI) Global Challenges Research Fund (GCRF) aimed to address global challenges to achieve the United Nations (UN) Sustainable Development Goals through 12 interdisciplinary research hubs. This research documents key lessons learned around working with Theory of Change (ToC) to guide Monitoring, Evaluation and Learning (MEL) within these complex research for development hubs. Interviews and document reviews were conducted in ten of the research hubs. The results revealed that only one hub invested in an explicit visual system mapping approach, and that funder timelines, budget constraints and issues with capacity and expertise limited the application of these approaches across all hubs. In contrast, many hubs attempted to deal with visual complexity by means of ether constructing multiple, nested ToCs, or a conscious simplification of complexity through reducing their ToC towards a straightforward and uncomplicated chain model or spherical model. While the former approach had some value, most hubs struggled to find capacity to support the full articulation of nested ToCs. In contrast, the latter approach resulted in ToCs which lacked detail or mechanism articulation, but which nevertheless were often ‘fit for purpose’ in ensuring effective communication and coherence across diverse stakeholders and sub-projects. We conclude that in instances where the reporting, funding and management cycles of complex research for development programmes cannot be adapted to properly support learning-based approaches to ToC development, imposing simplicity in the ToC might be fit for purpose. This might also be preferable to more complex visual approaches that are only partially realised.

What can be said about effect of an intervention without a control group? The lack of evaluative evidence is a long-standing problem for regulatory policies against work-related health and safety risks. The European Union Occupational Safety and Health Framework has been in operation for three decades and covers more than 200 million workers, but the most recent evaluation was inconclusive about the benefits generated by this framework. A theory-based evaluation focusing on mechanisms in combination with a design capturing within-intervention variations offers a way forward. The idea is to measure the prevalence of most likely mechanisms and their correlation with outcomes. This approach is illustrated in a large- N evaluation of the use of workplace assessments in the public sector in Denmark. The strengths and weakness of the workplace assessment legislation are assessed. It is shown how findings based on the presented approach contribute to the public debate about workplace assessments.