Therapist Self-Report of Fidelity to Core Elements of Family Therapy for Adolescent Behavior Problems: Psychometrics of a Pragmatic Quality Indicator Tool

Hogue, Aaron; Bobek, Molly; Porter, Nicole; MacLean, Alexandra; Bruynesteyn, Lila; Jensen‐Doss, Amanda; Henderson, Craig E.

doi:10.1007/s10488-021-01164-0

Cited by 11 publications

(5 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also consistent with prior research in this area (e.g., Caron et al, 2020 ), providers who were less competent in CPS tended to overestimate their global integrity. Nevertheless, the tradeoff in self-report accuracy may be worth the cost-savings (see Table 6 ), which would be in line with the growing trend of using providers’ self-report for pragmatic fidelity evaluation (e.g., Hogue, Bobek, et al, 2022 ; Hogue et al, 2015 ).…”

Section: Discussionmentioning

confidence: 70%

“…Given that a component may not occur in each session (the median occurrence of the 11 components ranged from 45% to 57% across forms and is the lowest in TAU sessions), paired t -tests were used to examine the differences in each component. Following prior practice ( Hogue, Bobek, et al, 2022 ), we adjusted the alpha level to p < .01 to reduce the likelihood of Type I error in these component-wise analyses.…”

Section: Methodsmentioning

confidence: 99%

“…A self-report approach, which does not require recording sessions or additional staff time for observation, would be particularly efficient and likely more acceptable to practitioners and clients. However, previous research suggests that providers’ self-reports of fidelity often lack concordance with independent observers who are trained to use a manualized coding system (e.g., Caron et al, 2020 ; Herschell et al, 2020 ; Hurlburt et al, 2010 ; McLeod et al, 2022 ; for exceptions, see Hogue, Bobek, et al, 2022 ; Hogue et al, 2015 ). Many factors may contribute to this lack of concordance, including providers’ attribution biases, inaccurate recall, or lack of insight or fidelity to the EBP, which can lead to poor identification of their fidelity errors ( Brosan et al, 2008 ; Caron et al, 2020 ).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Pragmatic fidelity measurement in youth service settings

Wang

Stoll

Eddy

et al. 2023

Implementation Research and Practice

View full text Add to dashboard Cite

Background Fidelity measurement is critical for developing, evaluating, and implementing evidence-based treatments (EBTs). However, traditional fidelity measurement tools are often not feasible for community-based settings. We developed a short fidelity rating form for the Collaborative Problem Solving (CPS) approach from an existing manualized coding system that requires extensive training. We examined the reliability and accuracy of this short form when completed by trained observers, untrained observers, and self-reporting providers to evaluate multiple options for reducing barriers to fidelity measurement in community-based settings. Methods Community-based treatment providers submitted recordings of youth service sessions in which they did, or did not, use CPS. For 60 recordings, we compared short-form fidelity ratings assigned by trained observers and untrained observers to those provided by trained observers on the manualized coding system. For 141 recordings, we compared providers’ self-reported fidelity on the short form to ratings provided by trained observers on the manualized coding system and examined providers’ accuracy as a function of their global fidelity. Results & Conclusions: The short form was reliable and accurate for trained observers. An assigned global integrity score and a calculated average of component scores on the short form, but not component scores themselves, were reliable and accurate for observers who had CPS expertise but no specific training on rating CPS fidelity. When providers self-reported fidelity on the short form, their global integrity score was a reliable estimate of their CPS integrity; however, providers with better CPS fidelity were most accurate in their self-reports. We discuss the costs and benefits of these more pragmatic fidelity measurement options in community-based settings.

show abstract

Section: Discussionmentioning

confidence: 70%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Pragmatic fidelity measurement in youth service settings

Wang

Stoll

Eddy

et al. 2023

Implementation Research and Practice

View full text Add to dashboard Cite

show abstract

“…The checklist also logs (1) which implementation scenario was selected for intervention delivery: pediatrician intervention during visit, pediatrician follow-up visit, or behavioral specialist intervention on the visit day; (2) whether the intervention occurred in person or via video call; and (3) the number of minutes spent delivering the intervention or interventions. Research indicates that clinicians can reliably self-report on the delivery of behavioral interventions with families using brief postencounter fidelity checklists [ 74 - 76 ]. In addition, during initial follow-up assessment, adolescents and caregivers report on which components they experienced during the encounter: psychoeducation, BNI, facilitated conversation, or RT.…”

Section: Methodsmentioning

confidence: 99%

Standard Versus Family-Based Screening, Brief Intervention, and Referral to Treatment for Adolescent Substance Use in Primary Care: Protocol for a Multisite Randomized Effectiveness Trial

Hogue,

Porter,

Ozechowski

et al. 2024

JMIR Res Protoc

Self Cite

View full text Add to dashboard Cite

Background Screening, brief intervention, and referral to treatment for adolescents (SBIRT-A) is widely recommended to promote detection and early intervention for alcohol and other drug (AOD) use in pediatric primary care. Existing SBIRT-A procedures rely almost exclusively on adolescents alone, despite the recognition of caregivers as critical protective factors in adolescent development and AOD use. Moreover, controlled SBIRT-A studies conducted in primary care have yielded inconsistent findings about implementation feasibility and effects on AOD outcomes and overall developmental functioning. There is urgent need to investigate the value of systematically incorporating caregivers in SBIRT-A procedures. Objective This randomized effectiveness trial will advance research and scope on SBIRT-A in primary care by conducting a head-to-head test of 2 conceptually grounded, evidence-informed approaches: a standard adolescent-only approach (SBIRT-A-Standard) versus a more expansive family-based approach (SBIRT-A-Family). The SBIRT-A-Family approach enhances the procedures of the SBIRT-A-Standard approach by screening for AOD risk with both adolescents and caregivers; leveraging multidomain, multireporter AOD risk and protection data to inform case identification and risk categorization; and directly involving caregivers in brief intervention and referral to treatment activities. Methods The study will include 2300 adolescents (aged 12-17 y) and their caregivers attending 1 of 3 hospital-affiliated pediatric settings serving diverse patient populations in major urban areas. Study recruitment, screening, randomization, and all SBIRT-A activities will occur during a single pediatric visit. SBIRT-A procedures will be delivered digitally on handheld tablets using patient-facing and provider-facing programming. Primary outcomes (AOD use, co-occurring behavior problems, and parent-adolescent communication about AOD use) and secondary outcomes (adolescent quality of life, adolescent risk factors, and therapy attendance) will be assessed at screening and initial assessment and 3-, 6-, 9-, and 12-month follow-ups. The study is well powered to conduct all planned main and moderator (age, sex, race, ethnicity, and youth AOD risk status) analyses. Results This study will be conducted over a 5-year period. Provider training was initiated in year 1 (December 2023). Participant recruitment and follow-up data collection began in year 2 (March 2024). We expect the results from this study to be published in early 2027. Conclusions SBIRT-A is widely endorsed but currently underused in pediatric primary care settings, and questions remain about optimal approaches and overall effectiveness. In particular, referral to treatment procedures in primary care remains virtually untested among youth. In addition, whereas research strongly supports involving families in interventions for adolescent AOD, SBIRT-A effectiveness trial testing approaches that actively engage family members in primary care are absent. This trial is designed to help fill these research gaps to inform the critical health decision of whether and how to include caregivers in SBIRT-A activities conducted in pediatric primary care. Trial Registration ClinicalTrials.gov NCT05964010; https://www.clinicaltrials.gov/study/NCT05964010 International Registered Report Identifier (IRRID) PRR1-10.2196/54486

show abstract

“…The RORSY protocol is intended to provide one such roadmap for this more flexible approach. Though still in developmental stages, initial efforts to develop online training procedures in core relationship-oriented treatment techniques for youth SUD have produced gains in clinician reliability and accuracy when coding video vignettes for technique use [95,96] and when self-reporting on delivery of relationship-oriented techniques with their own cases [97]. Overall, online learning management systems for training community clinicians in the full spectrum of evidence-based behavioral interventions are advancing in waves, though it remains to be seen when, how, and for whom such methods meaningfully augment clinician performance and client outcomes [98].…”

Section: Making Relationships a Priority For Youth With Oud: Practice...mentioning

confidence: 99%

Untitled

2023

AAD

View full text Add to dashboard Cite

This article introduces the Relationship-Oriented Recovery System for Youth (RORSY) protocol, which is designed to increase uptake of Medications for Opioid Use Disorder (MOUD) and related services among adolescents and young adults. Youth exhibit alarmingly poor rates of MOUD initiation and adherence, OUD services involvement and long-term recovery success. RORSY attends to three developmentally unique recovery needs of this age group: assess and bolster youth recovery capital, prioritize involvement of concerned significant others, and use digital direct-to-consumer recovery supports. RORSY contains five evidence-informed intervention modules that can be flexibly tailored to meet the individual and relationship needs of a given youth: Relational Orientation, Youth Recovery Management Planning, Relational Recovery Management Planning, Relationship Skills Building, and Digital Recovery Support Planning. The article concludes with practice and policy recommendations for making relationship-building a top clinical priority for addressing youth OUD.

show abstract

Therapist Self-Report of Fidelity to Core Elements of Family Therapy for Adolescent Behavior Problems: Psychometrics of a Pragmatic Quality Indicator Tool

Cited by 11 publications

References 58 publications

Pragmatic fidelity measurement in youth service settings

Pragmatic fidelity measurement in youth service settings

Standard Versus Family-Based Screening, Brief Intervention, and Referral to Treatment for Adolescent Substance Use in Primary Care: Protocol for a Multisite Randomized Effectiveness Trial

Untitled

Contact Info

Product

Resources

About