Monitoring and evaluation for thinking and working politically

Abstract

This article explores the challenges of monitoring and evaluating politically informed and adaptive programmes in the international development field. We assess the strengths and weaknesses of some specific evaluation methodologies which have been suggested as particularly appropriate for these kinds of programmes based on scholarly literature and the practical experience of the authors in using them. We suggest that those methods which assume generative causality are particularly well suited to the task. We also conclude that factoring in the politics of uncertainty and evidence generation and use is particularly important in order to recognize and value diverse experiential knowledge, integrate understandings of the local context, accommodate adaptation and realistically grapple with the power relations which are inherent in evaluation processes.

A summative content analysis of how programmes to improve the right to sexual and reproductive health address power

“…This broader literature of descriptive cross-sectional studies typically draw on paradigms from the social sciences, such as interpretivist and constructivist framings, that can enrich biomedical research by foregrounding social relations and constellations of causes of change. Such framings are often germane to programmes that aim to shift power relations 129 130. By contrast, the use of traditional public health measurement approaches to assess structural change, as more typically used in the programmes reviewed in this study, entails many challenges, including how to account for complexity and context and how to create measures that detect changes in power relations 12 25 131–137.…”

Evaluating programming that thinks and works politically: Challenges and emerging practice

“…The issue is not to constrain variables to identify any changes that result from the program but rather to review the program's contributions to change by interrogating through stakeholder engagement. As cited in Aston et al (2022), Generative logic and causality based on open systems and taking context and power relations into account are central to [TWP] programs in practice. This differs from experimental counterfactual logic, which is based on closed systems, controlling for context, and often disregards power (Greenhalgh & Manzano, 2021).…”

The Art and Craft of Bricolage in Evaluation

“…In qualitative research, the notion of bricolage has long been understood as a strategy that adds rigour (Denzin and Lincoln 1999;Chambers 2015). In practice, although many evaluators are bricoleurs, the dogma of evaluation 'brands' and notions of a supposed 'gold standard' have until recently hindered bricolage from gaining attention in the evaluation field (Patton 2019;Hargreaves 2021;Aston et al 2021).…”

“…Critical to combining is a need to look across methods (or parts of them) to ascertain how robust conclusions are likely to be from any particular mixture. Aston et al (2021) argue that for complexity-aware and learning-oriented programming, rigour can best be understood through the following criteria:…”