A multidimensional approach to assessing intervention fidelity in a process evaluation of audit and feedback interventions to reduce unnecessary blood transfusions: a study protocol

Lorencatto, Fabiana; Gould, Natalie; McIntyre, S.; During, Camilla; Bird, Jon; Walwyn, Rebecca; Cicero, Robert; Glidewell, Liz; Hartley, Suzanne; Stanworth, Simon; Foy, Robbie; Grimshaw, Jeremy; Michie, Susan; Francis, Jill J

doi:10.1186/s13012-016-0528-x

Cited by 42 publications

(40 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In keeping with recommendations for triangulation [ 38 , 39 ], findings regarding barriers and facilitators of logging in and using the SAR and regarding primary and secondary trial outcomes will be summarized in 1 table, known as a convergence coding matrix. Convergence of findings across all data sets will be assessed: full agreement (data convergence), partial agreement (complementarity between data), conflicting findings (discord) or silence (finding identified in only 1 data source and no additional sources) [ 40 ]. We will examine points of divergence to better understand the relationship between theory and outcomes, find out how an intervention may work or not in particular conditions, and identify areas of opportunity for future research on how interventions could be improved.…”

Section: Methodsmentioning

confidence: 99%

Testing Behavior Change Techniques to Encourage Primary Care Physicians to Access Cancer Screening Audit and Feedback Reports: Protocol for a Factorial Randomized Experiment of Email Content

Vaisson¹,

Witteman²,

Bouck³

et al. 2018

JMIR Res Protoc

View full text Add to dashboard Cite

BackgroundCancer Care Ontario’s Screening Activity Report (SAR) is an online audit and feedback tool designed to help primary care physicians in Ontario, Canada, identify patients who are overdue for cancer screening or have abnormal results requiring follow-up. Use of the SAR is associated with increased screening rates. To encourage SAR use, Cancer Care Ontario sends monthly emails to registered primary care physicians announcing that updated data are available. However, analytics reveal that 50% of email recipients do not open the email and less than 7% click the embedded link to log in to their report.ObjectiveThe goal of the study is to determine whether rewritten emails result in increased log-ins. This manuscript describes how different user- and theory-informed messages intended to improve the impact of the monthly emails will be experimentally tested and how a process evaluation will explore why and how any effects observed were (or were not) achieved.MethodsA user-centered approach was used to rewrite the content of the monthly email, including messages operationalizing 3 behavior change techniques: anticipated regret, material incentive (behavior), and problem solving. A pragmatic, 2x2x2 factorial experiment within a multiphase optimization strategy will test the redesigned emails with an embedded qualitative process evaluation to understand how and why the emails may or may not have worked. Trial outcomes will be ascertained using routinely collected administrative data. Physicians will be recruited for semistructured interviews using convenience and snowball sampling.ResultsAs of April 2017, 5576 primary care physicians across the province of Ontario, Canada, had voluntarily registered for the SAR, and in so doing, signed up to receive the monthly email updates. From May to August 2017 participants received the redesigned monthly emails with content specific to their allocated experimental condition prompting use of the SAR. We have not yet begun analyses.ConclusionsThis study will inform how to communicate effectively with primary care providers by email and identify which behavior change techniques tested are most effective at encouraging engagement with an audit and feedback report.Trial RegistrationClinicalTrials.gov NCT03124316; https://clinicaltrials.gov/ct2/show/NCT03124316 (Archived by WebCite at http://www.webcitation.org/6w2MqDWGu)

show abstract

Section: Methodsmentioning

confidence: 99%

Testing Behavior Change Techniques to Encourage Primary Care Physicians to Access Cancer Screening Audit and Feedback Reports: Protocol for a Factorial Randomized Experiment of Email Content

Vaisson¹,

Witteman²,

Bouck³

et al. 2018

JMIR Res Protoc

View full text Add to dashboard Cite

show abstract

“…This is currently a topic of great interest among KT researchers though there are no reports of evaluating implementation delity at a region or population level. 21 It may be of interest to integrate intervention implementation delity into the next iteration of our KT Signature Assessment Tool.…”

Section: Discussionmentioning

confidence: 99%

A method to audit and score implementation of knowledge translation (KT) interventions in large health regions – an observational pilot study using rectal cancer surgery in Ontario

Šimunović

Fahim

Coates

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: Across Ontario, since the year 2006 various knowledge translation (KT) interventions designed to improve the quality of rectal cancer surgery have been implemented by the provincial cancer agency or by individual researchers. Ontario is divided administratively into 14 health regions. We piloted a method to audit and score for each region of the province the KT interventions implemented to improve the quality of rectal cancer surgery. Methods: We interviewed stakeholders to audit KT interventions used in respective regions over years 2006 to 2014. Results were summarized into narrative and visual forms. Using a modified Delphi approach, KT experts reviewed these data and then, for each region, scored implementation of KT interventions using a 20-item KT Signature Assessment Tool. Scores could range from 20-100 with higher scores commensurate with greater KT intervention implementation. Results: There were thirty interviews. KT experts produced scores for each region that were bimodally distributed, with an average score for 2 regions of 78 (range 73-83) and for 12 regions of 30.5 (range 22-38). Conclusion: Our methods efficiently identified two groups with similar KT Signature scores. Two regions had relatively high scores reflecting numerous KT interventions and the use of sustained iterative approaches in addition to those encouraged by the provincial cancer agency, while 12 regions had relatively low scores reflecting minimal activities outside of those encouraged by the provincial cancer agency. These groupings will be used for future comparative quantitative analyses to help determine if higher KT signature scores correlate with improved measures for quality of rectal cancer surgery.

show abstract

“…This is currently a topic of great interest among KT researchers though there are no reports of evaluating implementation fidelity at a region or population level. 21 It may be of interest to integrate intervention implementation fidelity into the next iteration of our KT Signature Assessment Tool. Finally, scores for implementation of KT interventions at a region-level may not reflect impact on patient care, or the effectiveness of said interventions.…”

Section: Discussionmentioning

confidence: 99%

A method to audit and score implementation of knowledge translation (KT) interventions in large health regions – an observational pilot study using rectal cancer surgery in Ontario

Šimunović

Fahim

Coates

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: Across Ontario, since the year 2006 various knowledge translation (KT) interventions designed to improve the quality of rectal cancer surgery have been implemented by the provincial cancer agency or by individual researchers. Ontario is divided administratively into 14 health regions. We piloted a method to audit and score for each region of the province KT interventions implemented to improve the quality of rectal cancer surgery. Methods: We interviewed stakeholders to audit KT interventions used in respective regions over years 2006 to 2014. Results were summarized into narrative and visual forms. KT experts reviewed these data and then, for each region, scored KT intervention implementation using a 20-item KT Signature Assessment Tool. Scores could range from 20-100 with higher scores commensurate with greater KT intervention implementation. Results: There were thirty interviews. KT experts produced scores for each region that were bimodally distributed, with an average score for 2 regions of 78 (range 73-83) and for 12 regions of 30.5 (range 22-38). Conclusion: Our methods efficiently identified two groups with similar KT Signature scores. Two regions had relatively high scores reflecting numerous KT interventions and the use of sustained iterative approaches in addition to those encouraged by the provincial cancer agency, while 12 regions had relatively low scores reflecting minimal activities. These groupings will be used for future comparative quantitative analyses to help determine if higher KT signature scores correlate with improved measures for region-level quality of rectal cancer surgery.

show abstract

A multidimensional approach to assessing intervention fidelity in a process evaluation of audit and feedback interventions to reduce unnecessary blood transfusions: a study protocol

Cited by 42 publications

References 38 publications

Testing Behavior Change Techniques to Encourage Primary Care Physicians to Access Cancer Screening Audit and Feedback Reports: Protocol for a Factorial Randomized Experiment of Email Content

Testing Behavior Change Techniques to Encourage Primary Care Physicians to Access Cancer Screening Audit and Feedback Reports: Protocol for a Factorial Randomized Experiment of Email Content

A method to audit and score implementation of knowledge translation (KT) interventions in large health regions – an observational pilot study using rectal cancer surgery in Ontario

A method to audit and score implementation of knowledge translation (KT) interventions in large health regions – an observational pilot study using rectal cancer surgery in Ontario

Contact Info

Product

Resources

About