Non-formal educator use of evaluation results

Abstract

Increasing demands for accountability in educational programming have resulted in increasing calls for program evaluation in educational organizations. Many organizations include conducting program evaluations as part of the job responsibilities of program staff. Cooperative Extension is a complex organization offering non-formal educational programs through land grant universities. Many Extension services require non-formal educational program evaluations be conducted by field-based Extension educators. Evaluation research has focused primarily on the efforts of professional, external evaluators. The work of program staff with many responsibilities including program evaluation has received little attention. This study examined how field based Extension educators (i.e. program staff) in four Extension services use the results of evaluations of programs that they have conducted themselves. Four types of evaluation use are measured and explored; instrumental use, conceptual use, persuasive use and process use. Results indicate that there are few programmatic changes as a result of evaluation findings among the non-formal educators surveyed in this study. Extension educators tend to use evaluation results to persuade others about the value of their programs and learn from the evaluation process. Evaluation use is driven by accountability measures with very little program improvement use as measured in this study. Practical implications include delineating accountability and program improvement tasks within complex organizations in order to align evaluation efforts and to improve the results of both. There is some evidence that evaluation capacity building efforts may be increasing instrumental use by educators evaluating their own programs. Results indicate that there are few programmatic changes as a result of evaluation findings among the non-formal educators surveyed in this study. Extension educators tend to use evaluation results to persuade others about the value of their programs and learn from the evaluation process. Evaluation use is driven by accountability measures with very little program improvement use as measured in this study.Practical implications include delineating accountability and program improvement tasks within complex organizations in order to align evaluation efforts and to improve the results of both. There is some evidence that evaluation capacity building efforts may be increasing instrumental use by educators evaluating their own programs.

Useful to whom? Evaluation utilisation theory and boundaries for programme evaluation scope

“…18 Nevertheless, available research and evidence from evaluators' experiences suggest that persuasive utilisation is a common outcome of evaluation. 57,65 The prevalence of persuasive use may be linked to work-role expectations that emphasise the sharing of programme impacts with senior institutional leadership and programme stakeholders. 65 Correspondingly, it may be a valid by-product of evaluation in some contexts, when it is merited by evaluation findings.…”

“…57,65 The prevalence of persuasive use may be linked to work-role expectations that emphasise the sharing of programme impacts with senior institutional leadership and programme stakeholders. 65 Correspondingly, it may be a valid by-product of evaluation in some contexts, when it is merited by evaluation findings. However, when persuasive utilisation becomes the driving force for determining evaluation scope, then evaluation misuse is occurring.…”

“…Not only is it highly situational, 64 but it may often be guised as instrumental utilisation and leave few empirical traces 18 . Nevertheless, available research and evidence from evaluators’ experiences suggest that persuasive utilisation is a common outcome of evaluation 57,65 . The prevalence of persuasive use may be linked to work‐role expectations that emphasise the sharing of programme impacts with senior institutional leadership and programme stakeholders 65 .…”

Preparing Capable Youth Workers: The Project Youth Extension Service Approach

“…Intentional and synchronized programming, training, and evaluation plans (Little, 2014;Urban & Trochim, 2009;Walker, 2006) enhance program effectiveness. Engaging stakeholders in use of data processes and products (Ash & Clayton, 2009;Baughman, Boyd, & Franz, 2012) also fosters a critically-reflective learning community.…”

El proceso de evaluación en educación no formal: Un camino para su construcción

“…La evaluación es la acción de estimar esta conjugación de acciones para la rendición de cuentas, para determinar el impacto generado y, en ocasiones, para tomar decisiones. Es el ejercicio de otorgarle una calificación a este trabajo, en la mayoría de los casos con el fin de persuadir a los demás sobre su importancia (Baughman, Boyd, y Franz, 2012).…”