Exploring Stakeholder Engagement in Impact Evaluation Planning in Educational Development Work

Abstract

This article presents a case study of engaging stakeholders in the early stages of an impact evaluation of educational development work in a UK university. The rationale for undertaking participative impact evaluation is outlined in relation to the national and local context. The aim is to contribute to wider knowledge about appropriate methodology to lead to a better understanding of change processes in learning and teaching. We outline how stakeholder engagement in evaluation in this context has been influenced by the Aspen Institute's 'Theories of Change' approach, and how we interpreted and applied it in the context of a grant scheme for educational development work. Our experience is discussed in relation to previous learning about the application of the approach, in particular in the health sector. This has highlighted implications for future work, not least that it should be informed by a more appropriate theoretical framework for exploring the complexity of evaluation in this context. K E Y W O R D S : educational development; higher education; impact evaluation; stakeholder engagement; Theories of Change Evaluation

Rigour in Methods and Evaluation for Community Engagement

“…The integration of the term 'theory of change' into the logic model in the 1990s in the work of Carole Weiss and The Aspen Institute, has meant that there are a number of different ways that theory of change has been fused with specific evaluative tools such as program logic models. This is likely to be an irreversible process, for the concept of TOC is now so widespread that in fact some researchers have sought to lift out the 'participative' aspects of TOC for impact evaluation (Hart et al, 2009). Just as Love and Weiss have contended, it is argued here that theo-ry based approaches are not at all incongruent with community based interventions, because in-formal theory can emerge from community at various stages -but certainly some form of fun-damental theory is necessary (Love, 2012;Weiss, 1995).…”

Capturing the realities of sports programmes: systematic ‘messiness’?

“….a natural process in program development and implementation'. Hart et al (2009) also emphasised that evaluation had to be perceived as something that was done with a programme rather than done to it. They rationalised this approach by embracing Weiss' (1972) philosophy that participatory research is theory based, as it would attempt to explain how and why programmes might work as well as attempting to understand the stakeholder values and context.…”

The 3M National Teaching Fellowship: A High Impact Community of Practice in Higher Education

“…Future research might explore ways of how the fellowship program could address this challenge, considering the program does not engender a simple acceptance of these attitudes. We also encourage future researchers to consider the impact of the program from a broader perspective, perhaps seeking to examine the opinions of individuals outside of the fellowship program (e.g., Ahmad et al, 2017) or using other theoretical frameworks to examine impact (e.g., Guskey, 2002;Hart, Diercks-O'Brien, & Powell, 2009;Kirkpatrick, 1994).…”