Developmental evaluation and the ‘Stronger Economies Together’ initiative in the United States

Honadle, Beth Walter; Zapata, Marisa; Auffrey, Christopher; Hofe, Rainer vom; Looye, Johanna W.

doi:10.1016/j.evalprogplan.2013.11.004

Cited by 11 publications

(6 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our DE supported the engagement of stakeholders with CQI data to inform efforts to achieve multi-level system improvement in PHC systems for Indigenous people. Developmental evaluators are typically engaged as participant observers who guide data collection, inquiry and reflection-in-action [37, 51]. We had a team member who was able to undertake this role.…”

Section: Discussionmentioning

confidence: 99%

Using developmental evaluation to support knowledge translation: reflections from a large-scale quality improvement project in Indigenous primary healthcare

Laycock

Bailie

Matthews

et al. 2019

Health Res Policy Sys

View full text Add to dashboard Cite

Background Developmental evaluation is a growing area of evaluation practice, advocated for informing the adaptive development of change initiatives in complex social environments. The utilisation focus, complexity perspective and systems thinking of developmental evaluation suggest suitability for evaluating knowledge translation initiatives in primary healthcare. However, there are few examples in the literature to guide its use in these contexts and in Indigenous settings. In this paper, we reflect on our experience of using developmental evaluation to implement a large-scale knowledge translation research project in Australian Aboriginal and Torres Strait Islander primary healthcare. Drawing on principles of knowledge translation and key features of developmental evaluation, we debate the key benefits and challenges of applying this approach to engage diverse stakeholders in using aggregated quality improvement data to identify and address persistent gaps in care delivery. Discussion The developmental evaluation enabled the team to respond to stakeholder feedback and apply learning in real-time to successfully refine theory-informed research and engagement processes, tailor the presentation of findings to stakeholders and context, and support the project’s dissemination and knowledge co-production aim. It thereby contributed to the production of robust, useable research findings for informing policy and system change. The use of developmental evaluation appeared to positively influence stakeholders’ use of the project reports and their responses to the findings. Challenges included managing a high volume of evaluation data and multiple evaluation purposes, balancing facilitative sense-making processes and change with task-focused project management, and lack of experience in using this evaluation approach. Use of an embedded evaluator with facilitation skills and background knowledge of the project helped to overcome these challenges, as did similarities observed between features of developmental evaluation and continuous quality improvement. Conclusion Our experience of developmental evaluation confirmed our expectations of the potential value of this approach for strengthening improvement interventions and implementation research, and particularly for adapting healthcare innovations in Indigenous settings. In our project, developmental evaluation successfully encompassed evaluation, project adaptation, capacity development and knowledge translation. Further work is warranted to apply this approach more widely to improve primary healthcare initiatives and outcomes, and to evaluate implementation research.

show abstract

Section: Discussionmentioning

confidence: 99%

Using developmental evaluation to support knowledge translation: reflections from a large-scale quality improvement project in Indigenous primary healthcare

Laycock

Bailie

Matthews

et al. 2019

Health Res Policy Sys

View full text Add to dashboard Cite

show abstract

“…The difference from the two traditional and structured approaches is that formative evaluation focuses mostly on delivery and implementation, while summative emphasises outcomes. DE instead, assesses the project in real time, as it is being conducted and provides feedback throughout the project (Honadle et al, 2014). Moreover, it is considered a more appropriate, evaluative approach when innovations are in early stages, when environments are either Sustainability in logistics higher education changing or are particularly complex, when organisational learning is emphasised, or when systems and not individuals are the change target (Fagen et al, 2011).…”

Section: Developmental Evaluation: Complex Environmentsmentioning

confidence: 99%

“…DE-based evaluative research has mainly focused on academic or non-academic learning programmes (Singer and Kusumoto, 2006; Fagen et al , 2011; Honadle et al , 2014), whether personal development or developmental programmes (Lutz et al , 2013; Honadle et al , 2014), government initiatives or projects like public awareness programmes (Fagen et al , 2011; Herrenkohl et al , 2014). Literature on DE focuses on describing the use of DE, discussing the when and how DE may be useful on different programmes or projects (Patton, 2011, 2012; Herrenkohl et al , 2014).…”

Section: Developmental Evaluation: Complex Environmentsmentioning

confidence: 99%

Sustainable development: a developmental evaluation of logistics higher education in the Sultanate of Oman based on two innovation approaches (the triple helix of innovation and innovation competencies)

Nikolaou

2021

HEED

View full text Add to dashboard Cite

PurposeThis research seeks to identify evidence of innovation elements in the implementation of a sustainable development ecosystem in the HE environment. For the purposes of this investigation, the use of developmental evaluation has been deemed appropriate to fully explore the depths of the topic.Design/methodology/approachThe research follows a qualitative approach of inductive reasoning. For the purpose of this developmental evaluation, the collection of information from several stakeholders has been pursued in the study. Both a semi-structured interview and documents analysis were used.FindingsDifferent awareness levels among logistics faculty members, while the incentives given are minimal. Availability of grants and lack of tenure are some of the reasons raised. Industry is not cooperative in providing placements/internships restricting students of industry experience. Internationalisation is slow and international collaborations limited. Limited freedom in topical discussions and their implications to learning.Research limitations/implicationsThe research has considered possible limitations and used other methods for triangulation of the findings.Practical implicationsLow awareness on the implementation of pedagogical approaches for innovation. Not all faculty can be innovative (owing to current practice) neither they are incentivised to be so. Government spending is very low on R&D – 0.136 of the GDP in 2016. The industry is not ready for University-Business Collaborations, therefore achieving a low theory to practice ratio for students.Originality/valueIn the context in which the research has taken place (HE in Oman), there has been no evaluation (and more so developmental evaluation) previously implemented. Additionally, a longitudinal study, integrated as part of an ESD system targeted to innovation could increase the innovation capacity of the country on the international innovation index.

show abstract

“…As such, typically DE does not depart from an immutable program theory or logic, but rather one that evolves in step with emergent needs, reflecting creativity in development and response to change (Fagen et al 2011;Patton 2011;Rogers 2008). All of this is not to suggest that formative and summative evaluation must remain separate from DE, but may in fact integrate with the latter once the evaluand becomes sufficiently stable (Dickson and Saunders 2014;Lam and Shulha 2015); for example, where DE has served in "preformative development" or the readying of an evaluand for traditional evaluation (Fagen et al 2011;Patton 2011;Honadle et al 2014). It is also important to note that the methods employed within DE can be any-quantitative, qualitative, or a combination thereof-but the primary criteria for their selection should be whether they are "utilization focused" or "in service to developmental use," implying that they will elicit information that will inform directly the development of the evaluand (Patton 2011, p. 25).…”

Section: What Is Developmental Evaluation?mentioning

confidence: 99%

A Call for the Evaluation of Web-Based Climate Data and Analysis Tools

VanderMolen

Wall

Daudert

2019

Bulletin of the American Meteorological Society

View full text Add to dashboard Cite

Researchers are producing an ever greater number of web-based climate data and analysis tools in support of natural resource research and management. Yet the apparent absence or underreporting of evaluation in the development of these applications has raised questions as to whether, by whom, and for what they are utilized, and, relatedly, whether they meet the rationale for their development. This paper joins recent efforts to address these questions by introducing one approach to evaluation—developmental evaluation—and reporting on its use in the evaluation of the Southwest Climate and Environmental Information Collaborative (SCENIC). A web interface under development at the Western Regional Climate Center, SCENIC provides access to climate data and analysis tools to environmental scientists in support of natural resource research and management in the southwestern United States. Evaluation findings highlight subtlety in the improvements necessary for ensuring a useful and usable application that could not have been ascertained in the absence of end-user feedback. We therefore urge researchers to systematically evaluate web-based climate data and analysis tools in the interest of ensuring their usefulness, usability, and fulfillment of the proposed rationale. In so doing, we recommend that researchers test and apply established evaluation frameworks, thereby engaging end users directly in the process of application development.

show abstract

Developmental evaluation and the ‘Stronger Economies Together’ initiative in the United States

Cited by 11 publications

References 19 publications

Using developmental evaluation to support knowledge translation: reflections from a large-scale quality improvement project in Indigenous primary healthcare

Using developmental evaluation to support knowledge translation: reflections from a large-scale quality improvement project in Indigenous primary healthcare

Sustainable development: a developmental evaluation of logistics higher education in the Sultanate of Oman based on two innovation approaches (the triple helix of innovation and innovation competencies)

A Call for the Evaluation of Web-Based Climate Data and Analysis Tools

Contact Info

Product

Resources

About