Developing quality fidelity and engagement measures for complex health interventions

Walton, Holly; Spector, Aimee; Williamson, Morgan P.; Tombor, Ildikó; Michie, Susan

doi:10.1111/bjhp.12394

Cited by 76 publications

(114 citation statements)

References 37 publications

(81 reference statements)

Supporting

Mentioning

108

Contrasting

Unclassified

Order By: Relevance

“…Measures To measure fidelity of delivery and engagement, reliable checklists (containing all intervention components) and coding guidelines that can be used by researchers, providers and participants were developed specifically for this study [35].…”

Section: ) Assessing Fidelity Of Delivery Of and Engagement With Pridementioning

confidence: 99%

“…To identify whether fidelity of delivery and engagement varied across participants, providers and sites, data were collected from all four sites. All providers (n = 12) and people with dementia/supporters (n = 34) were asked to complete fidelity checklists after each session (see [35] for response rates).…”

Section: ) Assessing Fidelity Of Delivery Of and Engagement With Pridementioning

confidence: 99%

“…The intervention contained both standardised components that were delivered to all participants and tailored components that were dependent on which topics participants chose. Details of intervention components are reported in the PRIDE fidelity checklist development paper [35].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Developing strategies to improve fidelity of delivery of, and engagement with, a complex intervention to improve independence in dementia: a mixed methods study

Walton

Spector

Roberts

et al. 2020

BMC Med Res Methodol

Self Cite

View full text Add to dashboard Cite

Background: It is important to evaluate fidelity of delivery and engagement during feasibility trials. However, there is little guidance on how to systematically develop strategies to improve implementation if problems arise. We aimed to: 1) Assess fidelity of delivery and engagement, 2) Identify factors influencing fidelity of delivery and engagement, and 3) Develop strategies to improve fidelity of delivery of, and engagement with, a complex intervention to improve independence in dementia, within a feasibility trial. Methods: A mixed methods evaluation of an intervention that aimed to improve independence in dementia. To assess fidelity of delivery and engagement, observation and self-report methods were used: 60% of audio-recorded intervention sessions were transcribed and reliably rated for fidelity. Providers (n = 12) and people with dementia/ supporters (n = 34) were asked to complete checklists after each session. Descriptive statistics were used to analyse the data. To identify factors influencing fidelity and engagement, one-to-one semi-structured interviews were conducted with providers (n = 8), people with dementia (n = 7) and supporters (n = 7). Thematic analysis and content analysis were used to analyse data. To develop strategies, we followed four steps proposed by the authors of the Behaviour Change Wheel (1. Understanding the behaviour, 2. Identifying intervention functions, 3. Specifying intervention content, 4. Identifying mode of delivery).

show abstract

Section: ) Assessing Fidelity Of Delivery Of and Engagement With Pridementioning

confidence: 99%

Section: ) Assessing Fidelity Of Delivery Of and Engagement With Pridementioning

confidence: 99%

See 1 more Smart Citation

Developing strategies to improve fidelity of delivery of, and engagement with, a complex intervention to improve independence in dementia: a mixed methods study

Walton

Spector

Roberts

et al. 2020

BMC Med Res Methodol

Self Cite

View full text Add to dashboard Cite

show abstract

“…The Conceptual Framework for Implementation Fidelity (CFIF) (30) was used to guide both the measurement of delity and understand factors affecting its delivery. The CFIF structure facilitated the development of measurement tools, including a delity checklist (12,(41)(42)(43), data collection (44) and analysis (45).…”

Section: Methodsmentioning

confidence: 99%

Can a Complex Vocational Rehabilitation Intervention Be Delivered With Fidelity? Fidelity Assessment in the Fresh Feasibility Trial

Holmes

Fletcher-Smith

Merchán-Baeza

et al. 2020

Preprint

View full text Add to dashboard Cite

Background:Determining whether complex rehabilitation interventions are delivered with fidelity is important as differences can occur between sites, therapists delivering the intervention and over time; threatening trial outcomes and increasing the risk of Type II and Type III errors. Aims: to (1) evaluate implementation fidelity of vocational rehabilitation delivered in FRESH, a multi-centre feasibility randomised controlled trial; and (2) understand factors affecting delivery. Methods:Mixed methods evaluation. Fidelity was measured quantitatively using intervention case report forms, fidelity checklists and clinical records. Qualitative data from mentoring records, interviews with the intervention therapists, participants and their employers and NHS staff at each site explored moderators of implementation fidelity. The quantitative and qualitative data informed data collection tools and analysis. Data were examined against a logic model and benchmarked against an earlier cohort study. Results:Analysis of 38 clinical records (one per participant), 699 content CRFs (42-248 per therapist) and 12 fidelity checklists suggest intervention was delivered as intended. The core intervention process was followed in each case despite therapist variation. Qualitative data from clinical records, fidelity checklists, 183 mentoring records and 38 interviews (4 therapists, 15 trial participants, 6 employers and 13 NHS staff) explained factors affecting intervention fidelity. Fidelity moderators were similar across sites. Facilitators included therapists’ community rehabilitation experience, expert mentoring, and ability to individually tailor intervention. Barriers involved a lack of access to NHS systems no backfill and limited line-manager support. Factors that both helped or hindered intervention delivery were effective communication with participants, intervention acceptability, participants’ changing needs and interagency working. Different fidelity measures answered different questions. Fidelity checklists determined whether intervention processes were followed and explained moderators. Interviews provided insights into acceptability. Adherence was determined by content forms. Mentoring records described implementation barriers and how they were overcome.Conclusions:Mixed methods fidelity assessments enable trialists to identify factors likely to affect intervention fidelity in a definitive trial and longer-term clinical implementation. Mentoring provided insight into engagement and fidelity deviations that could be addressed in real-time, facilitating fidelity and offering a window on trial processesTrial registration: ISRCTN Registry, ISRCTN38581822 (Registered: 02/01/2014) (https://doi.org/10.1186/ISRCTN38581822)

show abstract

“…Figure 1 presents the concepts of three fidelity areas, delivery, receipt and enactment, as defined by the NIH Behavior Change Consortium in 2004 (9). Engagement has later been used as an umbrella term to refer to participant behaviours (10), both to what extent a participant understands and can perform the required skills (receipt), and to what extent a participant can put the skills into practice in daily life (enactment) (8). This paper focuses on the enactment of BCTs, while acknowledging the importance of receipt as a key aspect of participant engagement in interventions.…”

Section: Introductionmentioning

confidence: 99%

Participants’ enactment of behavior change techniques: a call for increased focus on what people do to manage their motivation and behavior

Hankonen

2020

Health Psychology Review

View full text Add to dashboard Cite

Intervention effectiveness does not only depend on fidelity of intervention delivery, but also the enactment, or use, of behavior change techniques (BCTs) by the participants. For example, it is not sufficient that intervention provider prompts an intervention participant to self-monitor their physical activity, crucially, the participant must also enact self-monitoring. Theoretical and conceptual work integrating various strands of research into 'what a person can do for oneself' to change behavior is needed. This paper argues how this would aid in designing for, assessing, and promoting the use individuals' self-management techniques, and ultimately, our understanding of sustained behavior change. The recently published, integrative compendium of self-enactable techniques to change and selfmanage motivation and behavior can act as a useful starting point for this work. An increased focus on the enactment of BCTs would help clarify intervention processes, help explain trial outcomes, and potentially enhance intervention effectiveness.

show abstract

Developing quality fidelity and engagement measures for complex health interventions

Cited by 76 publications

References 37 publications

Developing strategies to improve fidelity of delivery of, and engagement with, a complex intervention to improve independence in dementia: a mixed methods study

Developing strategies to improve fidelity of delivery of, and engagement with, a complex intervention to improve independence in dementia: a mixed methods study

Can a Complex Vocational Rehabilitation Intervention Be Delivered With Fidelity? Fidelity Assessment in the Fresh Feasibility Trial

Participants’ enactment of behavior change techniques: a call for increased focus on what people do to manage their motivation and behavior

Contact Info

Product

Resources

About