Measuring Implementation of Evidence-Based Programs Targeting Young Children at Risk for Emotional/Behavioral Disorders

Sutherland, Kevin S.; McLeod, Bryce D.; Conroy, Maureen A.; Cox, Julia R.

doi:10.1177/1053815113515025

Cited by 43 publications

(55 citation statements)

References 72 publications

(146 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Therefore, the significant increase in adherence for teachers in the BEST in CLASS group, coupled with the small, but not significant, increase in competence, suggests that teachers in BEST in CLASS were delivering more of the BEST in CLASS practices than teachers in the comparison group, and at a minimum delivering them just as well. While the relationship between adherence, competence, and child outcomes remains largely unknown (Harn, Parisi, & Stoolmiller, 2013;Sutherland, McLeod, Conroy, & Cox, 2013), the measurement of adherence and competence in the BEST in CLASS efficacy study will allow us to ultimately compare the effects of these different dimensions of teacher implementation on child outcomes (Durlak & Dupre, 2008). While future work will examine the relationship between these different dimensions of treatment integrity and child outcomes, the measurement of both adherence and competence of teacher delivery is an important direction for the field.…”

Section: Discussionmentioning

confidence: 97%

Implementation Integrity of Practice-Based Coaching: Preliminary Results from the BEST in CLASS Efficacy Trial

Sutherland

Conroy

et al. 2014

School Mental Health

Self Cite

View full text Add to dashboard Cite

The purpose of this article is to describe the practice-based coaching model used in BEST in CLASS, a Tier-2 classroom-based intervention comprised of evidence-based instructional practices designed to prevent and ameliorate the chronic problem behaviors of young children at risk for the development of emotional/behavioral disorders. Following a description of the model, data from year two of an ongoing 4-year randomized control trial are presented that describe (a) the amount (i.e., dosage) of coaching teachers received during BEST in CLASS implementation, (b) the integrity with which coaches implemented the BEST in CLASS coaching model, and (c) subsequent teacher implementation of the BEST in CLASS strategies. Forty-eight (23 BEST in CLASS; 25 comparison) teachers and ten coaches participated in this descriptive study. Data indicate that following coaches' observations, teachers received approximately 30 min of practice-based coaching during coaching meetings each week of implementation, and integrity data indicate that coaches implemented the critical coaching skills during the coaching meetings with integrity. Adherence data indicate that teachers in the treatment group increased their extensiveness of the use of BEST in CLASS practices at both post-treatment and 1-month follow-up compared to comparison teachers; competence data indicated that teachers in the treatment group increased the quality of delivery of practices at post-treatment compared to comparison teachers. Implications of these findings for both future research and practice-based coaching implementation are discussed.

show abstract

Section: Discussionmentioning

confidence: 97%

Implementation Integrity of Practice-Based Coaching: Preliminary Results from the BEST in CLASS Efficacy Trial

Sutherland

Conroy

et al. 2014

School Mental Health

Self Cite

View full text Add to dashboard Cite

show abstract

“…The emergence of implementation science (Eccles & Mittman, 2006) has highlighted variables within organizational systems, such as schools, that establish implementation as multidimensional (Wong, Ruble, McGrew, & Yu, 2018) or multilevel (Stahmer, Suhrheinrich, Schetter, & Hassrick, 2018). One dimension of implementation, as noted previously, consists of delivery of the treatment with fidelity (O'Donnell, 2008), with some researchers proposing that measurement of fidelity itself consists of multiple dimensions (Sutherland, McLeod, Conroy, & Cox, 2013). Other researchers note that implementation of a program consists of not only the fidelity of delivery or adherence to the program but also the supports that are necessary for the provider to achieve fidelity.…”

Section: Csesa Program Implementation Rationale and Assessmentmentioning

confidence: 99%

Moving Beyond Fidelity: Assessing Implementation of a Comprehensive Treatment Program for Adolescents With Autism Spectrum Disorder

Steinbrenner

Odom

Hall

et al. 2019

Exceptional Children

View full text Add to dashboard Cite

Assessing implementation of comprehensive treatment programs is a multifaceted process that should extend beyond measuring solely treatment fidelity. The purpose of this article is to describe and demonstrate a thorough process for assessing implementation and receipt of a comprehensive treatment program for students with autism spectrum disorders in high schools. The Center on Secondary Education for Students With Autism Spectrum Disorders (CSESA) developed a profile and index approach to assess implementation of seven features related to the CSESA intervention process and content. In a cluster-randomized trial, the CSESA implementation profile captured information about diverse features of implementation, and this index successfully differentiated between intervention and control schools. A multifeatured implementation profile and index is an important step forward for examining and implementing comprehensive treatment programs in educational settings.

show abstract

“…Mowbray, Holter, Teague, and Bybee (2003) suggest that fidelity measurement begins by identifying both structural and instructional ingredients in a treatment, establishing operational definitions and determining the best method of measurement. Previous research supports a multimethod approach to fidelity, capturing both the quality of the teacher's implementation and a quantitative view of the teacher's adherence to the delivery process (Dane & Schneider, 1998;Odom et al, 2010;Power et al, 2005;Sutherland, McLeod, Conroy, & Cox, 2013) with reliable and valid measures (Smith, Daunic, & Taylor, 2007). Including a number of different methods and multiple data collection points requires extra time and resources to complete data collection related to fidelity of implementation.…”

Section: Challenges To Measuring Fidelitymentioning

confidence: 99%

Fidelity of a Teacher-Implemented Intervention for Preschoolers With Autism Spectrum Disorder: No, Some, and Unexpected Effects

Reszka

Belardi

Amsbary

et al. 2019

Topics in Early Childhood Special Education

View full text Add to dashboard Cite

Advancing Social-Communication and Play (ASAP) is a classroom-based intervention targeting the social-communication and play development of preschool-age children with autism spectrum disorder. We used a multimethod approach to measuring intervention fidelity, including adherence, exposure, treatment differentiation, and quality of treatment delivery. Overall, teachers in the ASAP classrooms increased their adherence to the ASAP intervention compared with businessas-usual teachers. Teacher characteristics did not affect the ability to implement ASAP with fidelity. There were no group differences in coded video instructional quality scores. Differences in child engagement between the ASAP and businessas-usual groups were not mediated by fidelity adherence scores, contrary to our hypothesis. Finally, end-of-year summary of treatment delivery ratings did not align with adherence scores. Continued research regarding fidelity of school-based interventions should focus on distinguishing active ingredients from general instructional practices and learning from teachers about the child-level factors they consider when implementing an intervention. Keywords fidelity, ASAP, school interventions, autism spectrum disorder Research Question 1: Does a multimethod approach measure fidelity to ASAP and differentiate ASAP from business-as-usual (BAU) classrooms? Research Question 2: Are there student or teacher characteristics that moderate intervention fidelity? Research Question 3: Does ASAP treatment fidelity mediate student outcomes? Method Design This IES-funded Goal 3 efficacy trial, a multisite CRT, was conducted in public preschool classrooms in Florida, North

show abstract

Measuring Implementation of Evidence-Based Programs Targeting Young Children at Risk for Emotional/Behavioral Disorders

Cited by 43 publications

References 72 publications

Implementation Integrity of Practice-Based Coaching: Preliminary Results from the BEST in CLASS Efficacy Trial

Implementation Integrity of Practice-Based Coaching: Preliminary Results from the BEST in CLASS Efficacy Trial

Moving Beyond Fidelity: Assessing Implementation of a Comprehensive Treatment Program for Adolescents With Autism Spectrum Disorder

Fidelity of a Teacher-Implemented Intervention for Preschoolers With Autism Spectrum Disorder: No, Some, and Unexpected Effects

Contact Info

Product

Resources

About