Evaluation in dynamic times: Skateboard, pushbike, or quad bike?

Abstract

The turbulent and fluid environment in which we find ourselves due to the COVID-19 pandemic requires evaluative responses that facilitate learning, adaptation, and timeliness. This article examines the last of these—the need for timely evaluative information. Such information requires evaluators and their clients making trade-offs between what is desirable and what is feasible in a constrained time frame. Applying a light-hearted analogy—skateboard, pushbike, quad bike—three different evaluative approaches and the trade-offs that each involves are described. The notion of adequacy for purpose is then examined in terms of two dimensions: the level of certainty of the evidence that is required by the client, and the level of confidence required by the client in the evaluative claim/conclusion. The article demonstrates the need for evaluators to ensure their clients and other users of evaluative information understand the level of confidence and certainty that can be placed in it.

Pivoting to online: The benefits, challenges and possibilities for international programme evaluations

“…Evaluators as pragmatic professionals understand compromise is always needed but the approach taken should be valid for the purposes for which the evaluation is to be used (Nunns, 2020). Several clear learnings came from this work.…”

“…In any evaluation, compromises must be made between what is desirable and what is feasible (Nunns, 2020). The advent of COVID-19 altered the core principles of social science research and evaluation with travel and social distancing restrictions forcing evaluators and researchers to rethink their approach to field research (Howlett, 2022).…”

Rapid evaluation of service innovations in health and social care: key considerations

“…Rapid evaluation will lead to a greater range of uncertainty about likely outcomes, but it is important to embrace this and support decision-makers in accounting for such uncertainty. 68 It can be useful to explore the relationships between uncertain outcomes and the assumptions that are made. For example, an evaluation of different strategies for reopening schools after the first wave of the COVID-19 pandemic in the UK explored the relationships between different assumptions about the effectiveness of contract tracing and isolation on disease transmission.…”