Using process data to understand outcomes in sexual health promotion: an example from a review of school-based programmes to prevent sexually transmitted infections

Abstract

This article discusses how process indicators can complement outcomes as part of a comprehensive explanatory evaluation framework, using the example of skills-based behavioural interventions to prevent sexually transmitted infections and promote sexual health among young people in schools. A systematic review was conducted, yielding 12 eligible outcome evaluations, 9 of which included a process evaluation. There were few statistically significant effects in terms of changes in sexual behaviour outcomes, but statistically significant effects were more common for knowledge and self-efficacy. Synthesis of the findings of the process evaluations identified a range of factors that might explain outcomes, and these were organized into two overarching categories: the implementation of interventions, and student engagement and intervention acceptability. Factors which supported implementation and engagement and acceptability included good quality teacher training, involvement and motivation of key school stakeholders and relevance and appeal to young people. Factors which had a negative impact included teachers' failure to comprehend the theoretical basis for behaviour change, school logistical problems and omission of topics that young people considered important.It is recommended that process indicators such as these be assessed in future evaluations of schoolbased sexual health behavioural interventions, as part of a logic model.

Make a Move and Long Live Love+

“…The influence of these stakeholders has also been found in previous studies (Durlak & DuPre, 2008;Dusenbury, Brannigan, Falco, & Hansen, 2003;Paulussen, Kok, Schaalma, 1994;Shepherd et al, 2014;Wiefferink et al, 2005).…”

“…This may have influenced program outcomes negatively; e.g., fear appeals are only effective when self-efficacy for the advised behavior is high (Peters, Ruiter, & Kok, 2013;Ruiter, Kessels, Peters, & Kok, 2014). Previous research has shown that teachers' incomprehension of the theoretical basis for behavior change negative influences implementation (Buston et al, 2002;Shepherd et al, 2014). To avoid 'pick and choose', a clearer justification of the exercises may enhance fidelity and completeness, for example by elaborating on methods (not) used.…”

“…Process evaluations can give insight into how programs were delivered and can reveal how implementers and intervention audience experienced the intervention (De Visser et al, 2015;Durlak & DuPre, 2008;Moore et al, 2014;Shepherd et al, 2014). Findings from process evaluations can therefore serve multiple functions.…”

Cochrane Qualitative and Implementation Methods Group guidance series—paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings

“…Features/factors influencing implementation are examined one by one Using logic models/conceptual framework Factors influencing implementation of lay health worker programmes [44] a School-based behavioural interventions to prevent sexually transmitted infections [59] Contextual features from qualitative research are integrated holistically with findings on effectiveness to develop or refine the logic model Development of framework is flexible and can incorporate stakeholder input when description in published studies is poor Analysing programme theory School feeding programmes for disadvantaged children [54] Interventions to prevent repeat teenage pregnancy [64] a Contextual features from qualitative research are integrated holistically with effectiveness findings to develop and refine programme theory Expertise in programme theory required (e.g.realist evaluation) Testing hypotheses derived using subgroup analysis Promoting positive health behaviours [27] Hypotheses from qualitative synthesis can be tested statistically Requires sufficient numbers of trials to conduct sub-group analysis Features/factors influencing implementation are examined one by one Qualitative comparative analysis.…”

Implementation strategy for the school-based sex education program Long Live Love

“…Lack of awareness of the importance of completeness and fidelity was found to be related to incomplete program implementation of the online school-based sex education program, LLL+ for older high school students (van Lieshout et al, 2016). Previous research has shown that teachers' incomprehension of the theoretical basis for behavior change negatively influences implementation (Buston et al, 2002;Shepherd et al, 2014). Coaching can only commence when teachers develop an awareness of the need and desire to improve their performance or change the way they have been doing things at work (Parsloe & Leedham, 2009).…”