Toward a systemic approach to evaluating evaluation capacity development

Abstract

In this article, we contribute to evaluation capacity development (ECD) research and practice by demonstrating how a systems‐based evaluation approach can be applied to a complex ECD initiative. Since 2013, the German evaluation institute, DEval, has implemented an ECD program focused on Latin America and the Caribbean (LAC). This is a diverse, emergent program developed using a bottom‐up approach. DEval takes a facilitating rather than managerial role, supporting ECD among a range of stakeholders. The article discusses several issues related to evaluating this complex ECD program. The evaluation viewed the program as a system comprising two sub‐systems: one focused on improving the knowledge and skills of evaluators and stakeholders (capacity), and the other focused on fostering an enabling evaluation environment (capability). It drew on an evaluation tradition primarily focused on the consequences of evaluations. A focus on the consequences of evaluations is essentially one step beyond evaluation use (an instrumental orientation) toward usefulness (a value‐based orientation). The evaluation adopted a systemic approach to define the core boundaries of evaluation focus, acknowledging that various stakeholders perceive the evaluation differently based on their interests or stake. By aligning the evaluation with these perspectives, it could focus on what matters most to different stakeholder groups. Four key perspectives or framings were identified: the program itself, the transferability of the LAC lessons to ECD in other parts of the world, collaborations with global agencies undertaking ECD at a global level, and institutional practices within DEval.