A systematic review and reformulation of outcome evaluation in clinical supervision: Applying the fidelity framework.

Reiser, Robert P.; Milne, Derek

doi:10.1037/tep0000031

Cited by 45 publications

(48 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Components in the domains of provider training, treatment receipt and treatment enactment were infrequently reported. This is in contrast to reviews utilizing the NIH Fidelity Framework in other areas that found higher reporting of enactment . It should be noted that one enactment item (Reporting use of a strategy to improve subject performance of intervention skills) was excluded from this review, as adequate reliability could not be obtained.…”

Section: Discussionmentioning

confidence: 97%

Reporting of treatment fidelity in behavioural paediatric obesity intervention trials: a systematic review

et al. 2016

View full text Add to dashboard Cite

Behavioral interventions for pediatric obesity are promising, but detailed information on treatment fidelity (i.e., design, training, delivery, receipt, and enactment) is needed to optimize the implementation of more effective interventions. Little is known about current practices for reporting treatment fidelity in pediatric obesity studies. This systematic review, in accordance with PRISMA guidelines, describes the methods used to report treatment fidelity in randomized controlled trials. Treatment fidelity was double-coded using the NIH Fidelity Framework checklist. Three hundred articles (N=193 studies) were included. Mean inter-coder reliability across items was 0.83 (SD=0.09). Reporting of treatment design elements within the field was high (e.g., 77% of studies reported designed length of treatment session), but reporting of other domains was low (e.g., only 7% of studies reported length of treatment sessions delivered). Few reported gold standard methods to evaluate treatment fidelity (e.g., coding treatment content delivered). General study quality was associated with reporting of treatment fidelity (p<0.01) as was the number of articles published for a given study (p<0.01). The frequency of reporting treatment fidelity components has not improved over time (p=0.26). Specific recommendations are made to support pediatric obesity researchers in leading health behavior disciplines toward more rigorous measurement and reporting of treatment fidelity.

show abstract

Section: Discussionmentioning

confidence: 97%

Reporting of treatment fidelity in behavioural paediatric obesity intervention trials: a systematic review

et al. 2016

View full text Add to dashboard Cite

show abstract

“…Only 19.6% of the studies identified from the forward citation search to report on fidelity were found to address receipt, compared with 33% in a recent review on clinical supervision [85]. Amongst the studies identified, 60.6% assessed receipt in relation to understanding (compared to 0–69% in other reviews [10, 14–17]) and 42.4% in relation to performance of skill (39–65% in other reviews [10, 14–17]).…”

Section: Discussionmentioning

confidence: 99%

Methods used to address fidelity of receipt in health intervention research: a citation analysis and systematic review

Rixon

Baron

McGale

et al. 2016

BMC Health Serv Res

115

104

View full text Add to dashboard Cite

BackgroundThe American Behaviour Change Consortium (BCC) framework acknowledges patients as active participants and supports the need to investigate the fidelity with which they receive interventions, i.e. receipt. According to this framework, addressing receipt consists in using strategies to assess or enhance participants’ understanding and/or performance of intervention skills. This systematic review aims to establish the frequency with which receipt is addressed as defined in the BCC framework in health research, and to describe the methods used in papers informed by the BCC framework and in the wider literature.MethodsA forward citation search on papers presenting the BCC framework was performed to determine the frequency with which receipt as defined in this framework was addressed. A second electronic database search, including search terms pertaining to fidelity, receipt, health and process evaluations was performed to identify papers reporting on receipt in the wider literature and irrespective of the framework used. These results were combined with forward citation search results to review methods to assess receipt. Eligibility criteria and data extraction forms were developed and applied to papers. Results are described in a narrative synthesis.Results19.6% of 33 studies identified from the forward citation search to report on fidelity were found to address receipt. In 60.6% of these, receipt was assessed in relation to understanding and in 42.4% in relation to performance of skill. Strategies to enhance these were present in 12.1% and 21.1% of studies, respectively. Fifty-five studies were included in the review of the wider literature. Several frameworks and operationalisations of receipt were reported, but the latter were not always consistent with the guiding framework. Receipt was most frequently operationalised in relation to intervention content (16.4%), satisfaction (14.5%), engagement (14.5%), and attendance (14.5%). The majority of studies (90.0%) included subjective assessments of receipt. These relied on quantitative (76.0%) rather than qualitative (42.0%) methods and studies collected data on intervention recipients (50.0%), intervention deliverers (28.0%), or both (22.0%). Few studies (26.0%) reported on the reliability or validity of methods used.ConclusionsReceipt is infrequently addressed in health research and improvements to methods of assessment and reporting are required.Electronic supplementary materialThe online version of this article (doi:10.1186/s12913-016-1904-6) contains supplementary material, which is available to authorized users.

show abstract

“…Only 18 supervision-client outcome studies were conducted from 1981 to 2006 (Watkins, 2011a), and the vast majority of these studies were marked by several methodological shortcomings. Shortcomings included 1) lack of manuals operationalizing supervision, 2) insufficient documentation of supervisor training, 3) failure to demonstrate supervision integrity, 4) poorly defined and psychometrically weak instruments to measure clinician and client outcomes, 5) very small sample sizes of supervisors and clinicians reducing power for hypothesis testing, 6) inclusion of clinicians unrepresentative of community program service providers, and 7) no comparison or control conditions, random assignment, or routine follow-up assessments (Ellis & Ladany,1997; Ellis et al, 1996; Herschell et al, 2010; Holloway & Neufeldt, 1995; Reisner & Milne, 2014; Roth, Pilling, & Turner, 2010; Watkins, 2011a; Schoenwald et al, 2009). Moreover, none of the studies controlled for the effect of seminars or workshops that preceded supervision.…”

Section: Introductionmentioning

confidence: 99%

The Effectiveness and Cost of Clinical Supervision for Motivational Interviewing: A Randomized Controlled Trial

Martino

Paris

Añez

et al. 2016

Journal of Substance Abuse Treatment

View full text Add to dashboard Cite

The effectiveness of a competency-based supervision approach called Motivational Interviewing Assessment: Supervisory Tools for Enhancing Proficiency (MIA: STEP) was compared to Supervision-As-Usual (SAU) for increasing clinicians’ motivational interviewing (MI) adherence and competence and client retention and primary substance abstinence in a multisite Hybrid Type 2 effectiveness-implementation randomized controlled trial. Participants were 66 clinicians and 450 clients within one of eleven outpatient substance abuse programs. An independent evaluation of audio recorded supervision sessions indicated that MIA: STEP and SAU were highly and comparably discriminable across sites. While clinicians in both supervision conditions improved their MI performance, clinician supervised with MIA: STEP, compared to those in SAU, showed significantly greater increases in the competency in which they used fundamental and advanced MI strategies when using MI across seven intakes through a 16-week follow-up. There were no retention or substance use differences among the clients seen by clinicians in MIA: STEP or SAU. MIA: STEP was substantially more expensive to deliver than SAU. Innovative alternatives to resource-intensive competency-based supervision approaches such as MIA: STEP are needed to promote the implementation of evidence-based practices.

show abstract

A systematic review and reformulation of outcome evaluation in clinical supervision: Applying the fidelity framework.

Cited by 45 publications

References 43 publications

Reporting of treatment fidelity in behavioural paediatric obesity intervention trials: a systematic review

Reporting of treatment fidelity in behavioural paediatric obesity intervention trials: a systematic review

Methods used to address fidelity of receipt in health intervention research: a citation analysis and systematic review

The Effectiveness and Cost of Clinical Supervision for Motivational Interviewing: A Randomized Controlled Trial

Contact Info

Product

Resources

About