Supporting health promotion practitioners to undertake evaluation for program development

Abstract

BackgroundThe vital role of evaluation as integral to program planning and program development is well supported in the literature, yet we find little evidence of this in health promotion practice. Evaluation is often a requirement for organisations supported by public funds, and is duly undertaken, however the quality, comprehensiveness and use of evaluation findings are lacking. Practitioner peer-reviewed publications presenting evaluation work are also limited. There are few published examples where evaluation is conducted as part of a comprehensive program planning process or where evaluation findings are used for program development in order to improve health promotion practice.DiscussionFor even the smallest of programs, there is a diverse array of evaluation that is possible before, during and after program implementation. Some types of evaluation are less prevalent than others. Data that are easy to collect or that are required for compliance purposes are common. Data related to how and why programs work which could be used to refine and improve programs are less commonly collected. This finding is evident despite numerous resources and frameworks for practitioners on how to conduct effective evaluation and increasing pressure from funders to provide evidence of program effectiveness. We identify several organisational, evaluation capacity and knowledge translation factors which contribute to the limited collection of some types of data. In addition, we offer strategies for improving health promotion program evaluation and we identify collaboration of a range of stakeholders as a critical enabler for improved program evaluation.SummaryEvaluation of health promotion programs does occur and resources for how to conduct evaluation are readily available to practitioners. For the purposes of program development, multi-level strategies involving multiple stakeholders are required to address the organisational, capacity and translational factors that affect practitioners’ ability to undertake adequate evaluation.

Talking about the not talked about: use of, and reactions to, a DVD promoting bowel cancer screening to Aboriginal people

“…However, only a small number of evaluations of Indigenous health promotion tools, of generally moderate quality, have been undertaken 13 . While evaluation challenges are faced by all health promotion practitioners, 16 additional challenges may be experienced in the course of evaluating Indigenous health promotion tools due to issues typically present in rural/remote and Indigenous health settings. These issues include frequent workforce turnover; staff overloaded with multiple responsibilities (thus having limited capacity to pursue evaluations); limited funding; and short time‐frame of projects in general 17 .…”

Designing evaluation plans for health promotion mHealth interventions: a case study of the Milk Man mobile app

“…To generate evidence to improve health promotion practice, evaluation needs to be conducted throughout the implementation of an intervention not only at the conclusion. 20 A multifaceted evaluation plan is required to assess the complex web of components that could impact on the success of an mHealth intervention.…”

“…This encompassed three stages of evaluation: formative (takes place while planning interventions), process (takes place during the implementation) and impact (assesses the outcomes of the intervention) evaluation. 20,26 Milk Man is the first breastfeeding app we are aware of that targets fathers. The development of the app included formative evaluation with input from the target group and health professionals, and a usertesting phase encompassing a think-aloud walkthrough and completion of the MARS.…”

Skills, systems and supports: An Aboriginal Community Controlled Health Service (Apunipima) approach to building health promotion evaluation capacity of staff

“…Comprehensive evaluation requires time, appropriate skills of the evaluators, collaboration with project partners and stakeholders, and adequate resources . Building pertinent evaluation skills is a critical component of workforce development.…”

“…The first was staff evaluation knowledge and skills. The second was an organisational focus on achievement of a program's purpose rather than satisfying auditing requirements; and the third was translation and sharing of evaluation findings to shape future practice . Other studies have shown that staff trained in health promotion have difficulty translating new learnings into practice without organisational support .…”