Insights on Using Developmental Evaluation for Innovating

Abstract

This article contributes to research on evaluation by examining the capacity and contribution of developmental evaluation for innovating. This case study describes the preformative development of an educational program (from conceptualization to pilot implementation) and analyzes the processes of innovation within a developmental evaluation framework. Developmental evaluation enhanced innovation by (a) identifying and infusing data primarily within an informing process toward resolving the uncertainty associated with innovation and (b) facilitating program cocreation between the clients and the developmental evaluator. Analysis into the demands of innovation revealed the pervasiveness of uncertainty throughout development and how the rendering of evaluative data helped resolve uncertainty and propelled development forward. Developmental evaluation enabled a nonlinear, coevolutionary program development process that centered on six foci-definition, delineation, collaboration, prototyping, illumination, and reality testing. This article concludes by encouraging evaluators to understand the demands of innovation and the value of design thinking when innovating.

Supporting systems transformation through design‐driven evaluation

“…This shift in orientation to embedding evaluation as a part of the innovation itself further advances our understanding of the concept of adaptation in practice . Complexity and systems‐informed approaches like Developmental Evaluation recognize the critical role of learning, adaptive strategy in program development and innovation contexts (Dinca‐Panaitescu, 2020; Lam & Shulha, 2015). Yet, the knowledge, skills, and abilities of program staff to design under conditions of dynamic complexity are often assumed rather than fully articulated in these methods.…”

A Case Study of the Guiding Principles for Collaborative Approaches to Evaluation in a Developmental Evaluation Context

“…In our midpoint data, signifi cant conceptual confusion surrounding the diff erence between evaluation and other forms of inquiry was revealed. Distinguishing evaluation from research has been a focus of theorizing in the fi eld of evaluation for some time (e.g., Fain, 2005 ;Fitzpatrick, Sanders, & Worthen, 2004 ;Levin-Rozalis, 2003 ;Patton, 2014 ). Some participants attributed their initial thinking to past experiences where research was viewed as more esoteric and evaluation as a topdown activity that did not easily generate useful fi ndings.…”

Developmental evaluation: A tool to support innovation

“…Patton (2011) notes that the deep knowledge accumulated by an experienced evaluator about the nature of what is likely to work is very useful in helping a development team articulate an initiative's programme theory. Programme theory was used to provide a framework to inform the development of the Assessment Pilot Initiative (Lam & Shulha, 2015). In New Zealand, McKegg, Wehipeihana, Pipi and Thompson (2013) used a developmental evaluation approach to support the implementation of a significant change in focus for He Oranga Poutama.…”

“…During the developmental evaluation of the Māori and Pacific Educational Initiative, the evaluators found that instead of an approach of interim and final reports, the needs of the trustees of the philanthropic trust to be kept informed of progress were better met by regular, shorter reports containing quantitative data as it became available . Lam and Shulha (2015) consider that the nature of the data used in the development of a teacher education tool was broader than what is traditionally collected in a summative evaluation, such as data about an initiative's activities or outcomes. For example, the data provided included information from the social science and education literature about the use of technology to assist learning, which provided the development team with new ideas for the initiative.…”

“…The degree of power and control a development team is willing to give the evaluator will vary and this affects the extent to which the evaluator can influence the development (Lam & Shulha, 2015). A developmental evaluator often has to strike a balance between the detailed understanding and opportunities for influence that come with being part of the team and being independent and objective (Gamble, 2008).…”