Evaluating Policy, Systems, and Environmental Change Interventions: Lessons Learned From CDC’s Prevention Research Centers

Abstract

IntroductionThe field of public health is increasingly implementing initiatives intended to make policies, systems, and environments (PSEs) more supportive of healthy behaviors, even though the evidence for many of these strategies is only emerging. Our objective was 3-fold: 1) to describe evaluations of PSE-change programs in which the evaluators followed the steps of the Centers for Disease Control and Prevention’s (CDC’s) Framework for Program Evaluation in Public Health, 2) to share the resulting lessons learned, and 3) to assist future evaluators of PSE-change programs with their evaluation design decisions.MethodsSeven Prevention Research Centers (PRCs) applied CDC’s framework to evaluate their own PSE-change initiatives. The PRCs followed each step of the framework: 1) engage stakeholders, 2) describe program, 3) focus evaluation design, 4) gather credible evidence, 5) justify conclusions, and 6) ensure use and share lessons learned.ResultsEvaluation stakeholders represented a range of sectors, including public health departments, partner organizations, and community members. Public health departments were the primary stakeholders for 4 of the 7 evaluations. Four PRCs used logic models to describe the initiatives being evaluated. Their evaluations typically included both process and outcome questions and used mixed methods. Evaluation findings most commonly focused on contextual factors influencing change (process) and the adoption or implementation of PSE-change strategies (outcome). Evaluators shared lessons learned through various channels to reach local stakeholders and broader public health audiences.Conclusion Framework for Program Evaluation in Public Health is applicable to evaluations of PSE-change initiatives. Using this framework to guide such evaluations builds practice-based evidence for strategies that are increasingly being used to promote healthful behaviors.

A systematic review of components of community-based organisation engagement

“…The majority of studies ( n = 13) in this review were case studies. The remaining studies included quantitative design (Bloom et al., ; Chandra et al., ; Geist, ; Glik et al., ; Goytia et al., ; Patterson et al., ); qualitative design (Boutain & McNees, ; Burkhalter et al., ; Collignon, Men, & Tan, ; Honeycutt et al., ; Johnson et al., ; Radermacher, Karunarathna, Grace, & Feldman, ; Taras et al., ); systematic review (Frew et al., ); and mixed‐method design (Houston et al., ; Kakietek et al., ; Kelaher et al., ; Laborde, Brannock, Breland‐Noble, & Parrish, ; Riehman et al., ).…”

“…Identifying the need for a partnership enables CBOs to be actively involved in selecting best approaches to achieve the goals of a research project, and promotes successful partnerships (Akintobi et al., ; Bloom et al., ; McFarlane et al., ). CBOs can identify the appropriate participants; determine effective means of accessibility; and assist in the development of assessment tools and data collection methods (Frew et al., ; Geist, ; Goytia et al., ; Honeycutt et al., ; Radermacher et al., ; Taras et al., ).…”

“…Most studies agree that partnership dynamics are facilitated when CBOs are involved throughout the duration of the research project as opposed to one point (Geist, ; Kelaher et al., ; Napoles et al., ; Singer et al., ). A CBO can contribute to a partnership through several ways that include: planning, recruitment, developing instruments, data collection, data analysis and interpretation, dissemination of findings, intervention development and providing recommendations (Bloom et al., ; Boutain & McNees, ; Chaskin et al., ; Honeycutt et al., ; Houston et al., ). Depending on the proposed project, one or more of these levels will be more appropriate.…”

“…The point of contribution for a CBO is also important for engagement. For example, Honeycutt et al., reported that CBO engagement at the intervention phase of the research project ensured that findings were adequately utilised. CBOs are direct links to target communities they serve and can utilise findings from a study in the services they provide beyond the duration of the study or the partnership; they can also influence changes in policy and health outcomes of the target community (Honeycutt et al., ).…”

Engaging Fathers in Early Obesity Prevention During the First 1,000 Days: Policy, Systems, and Environmental Change Strategies

“…Building from the noted barriers to engagement, and utilizing a PSE approach , Table outlines possible strategies to increase father engagement in programs during the first 1,000 days and thereby the receipt of integrated obesity prevention services. A PSE approach emphasizes PSE factors that support or impede sustainable behaviors and health outcomes and highlights that health‐behavior change is unlikely to occur unless policies, systems, and environments make such change feasible .…”