Factors influencing implementation dose and fidelity thereof and related student outcomes of an evidence-based national HIV prevention program

Abstract

BackgroundTeachers’ implementation of evidence-based prevention programs in schools is inconsistent. Using data gathered from the national implementation among grade six students in The Bahamas of an evidence-based HIV intervention [Focus on Youth in the Caribbean (FOYC)], this study examines differences in the degree of implementation (“dose”) and adherence to the core activities (“fidelity of implementation”) by teachers according to theoretically and historically relevant teachers’ characteristics, attitudes, and experiences pre-intervention and post-intervention. The relationship of implementation dose and implementation fidelity is assessed according to student outcomes.MethodsBeginning in 2008, the Bahamian Ministry of Education (MOE) included FOYC in the grade six curriculum nationwide. Consistent with standard practice, teachers were offered MOE training workshops in FOYC prior to delivery. The MOE conducted an anonymous curricular assessment among the grade six students at the beginning and end of the school year. Teachers agreeing to participate in the research component were asked to complete a pre-implementation and post-implementation assessment of attitudes and prior experiences.ResultsTeachers taught 15.6 out of 30 core activities, 24 out of the 46 total activities, and 4.6 out of 8 sessions on average. Three teachers’ implementation groups were identified: 1) High Implementation Group (31.7% of the teachers), characterized by high levels of implementation dose and fidelity of implementation; 2) Moderate Implementation Group (52.8%), showing moderate levels of implementation dose but high levels of fidelity of implementation; and 3) Low Implementation Group (15.6%), with low levels of implementation dose and fidelity of implementation. Low Implementation Group teachers compared to teachers in the two higher performing groups had less training in interactive teaching, limited prior exposure to the FOYC curriculum, incomplete attendance at FOYC training workshops, and low levels of comfort in teaching FOYC lessons. Students taught by teachers in the Low Implementation Group demonstrated poorer outcomes relevant to the four student outcomes (HIV/AIDS knowledge, preventive reproductive health skills, self-efficacy, and intention to use protection if they were to have sex).ConclusionsBoth implementation dose and implementation fidelity are related to student outcomes. Teachers at risk for limited implementation can be identified pre-intervention, thus opening the possibility for focused pre-intervention training.

Teachers’ Patterns of Implementation of an Evidence-Based Intervention and Their Impact on Student Outcomes: Results from a Nationwide Dissemination over 24-Months Follow-Up

“…As described earlier in this manuscript, previously we have shown that grade 6 students taught by the one-third of teachers who were “high implementers” (characterized by high levels of sustained implementation and fidelity of implementation) and the one-half of teachers who were “moderate implementers” (characterized moderate levels of sustained implementation but high levels of fidelity of implementation) performed significantly better than students taught by the one-sixth of teachers who were “low implementers” (demonstrating poor implementation and fidelity [42]). To assess whether these differences in student performance with regard to reproductive health knowledge, skills, self-efficacy and intentions as a function of grade 6 teacher performance grouping persisted one year later, students within the three teacher groupings were compared using ANOVA with the Tukey HSD post hoc tests.…”

“…Per standard MOE procedure, both the grade 6 and grade 7 teachers were invited to participate in teacher training workshops; for the grade 6 teachers the training lasted for one to four days, for the grade 7, the training was one day. Details of the training provided to the teachers are described in our recent publications [41, 42]. …”

“…As reported previously [42], cluster analysis was used to identify teachers’ implementation patterns based on implementation dose (i.e., number of core activities completed) and fidelity of implementation (i.e., percentage of core activities being changed during the implementation). Results of the two-step cluster analysis indicated three distinct clusters of teachers: 1) High Implementation Group (32% of the teachers), characterized by high levels of sustained implementation and fidelity of implementation.…”

“…In our analyses conducted at the end of grade 6, student outcomes varied by the quality of implementation of their teachers; students taught by teachers in the Low Implementation Group demonstrated poorer outcomes in HIV/AIDS knowledge, preventive reproductive health skills and intention to use protection if they were to have sex) [42]. In this manuscript we seek to understand: 1) whether the youth continue to show improvement in HIV prevention skills over time (one year after the conclusion of the prevention intervention); 2) whether a brief national FOYC-booster program administered to all grade 7 students changes the outcome trajectory seen among grade 6 students based on grade 6 teacher FOYC- implementation patterns; and 3) whether differing implementation patterns among grade 7 students exerts an independent effect on student outcomes.…”

A Randomized-Controlled Trial of the Effects of Online Training on Implementation Fidelity

“…While rigorous experimental studies have documented the effectiveness of some programs at reducing HIV transmission, their full effects often are not realized in the real world due to challenges encountered by educators in implementing the programs (Collins, Harshbarger, Sawyer, & Hamdallah, 2006; Wang et al, 2015a, 2015b) and instructional support is warranted (Fagan, Hanson, Hawkins, & Arthur, 2008; Kam, Greenberg, & Walls, 2003). Schools are complex organizational settings and no evaluation can account for all the differences in structures, players, and cultures (Jaycox et al, 2006).…”