Assessing the Within‐Trial Treatment Integrity of Discrete‐Trial Teaching Programs Using Sequential Analysis

Abstract

Discrete-trial teaching is a strategy frequently used to teach functional skills to individuals with developmental and intellectual disabilities. Research has shown that the within-trial components of the procedure should be administered with ≥90% treatment integrity to facilitate optimal learning. Usually within-trial treatment integrity is measured using whole-session methods such as percentage of trials correctly administered. This study demonstrated one-step Markov transition matrices as a method of assessing within-trial treatment integrity. All components of discrete trials were coded and timestamped from video recordings of therapist-learner dyads in their typical setting (home or school). Several types of within-trial treatment integrity errors were identified using the Markov transition matrices, error sequences that could not be identified using a percentage correct analysis. Better identification of errors has the potential both to enhance treatment integrity and to gain efficiency by targeted retraining of therapists.Discrete trial teaching (DTT) is a strategy frequently used to teach functional skills to individuals with developmental and intellectual disabilities. DTT is a critical component of early intensive behavioral intervention programs. Most skills taught during the initial stages of early intensive behavioral intervention use a DTT format. DTT consists of five distinct within-trial components (Koegel, Russo, & Rincover, 1977). These components are the discriminative stimulus (S D ), a prompt if necessary, a response (correct, incorrect, or no response), a consequence determined by the type of learner response, and an intertrial interval (ITI). Discrete teaching trials have a distinct beginning and end. The presentation of the S D constitutes the beginning of the trial and the start of the ITI the end. In the early stages of teaching, DTT takes place in a structured and controlled environment consisting of one-to-one teaching sessions between therapists and learners. DTT involves breaking down particular skills into smaller parts and teaching each one of these smaller skills individually.utterance to indicate to the learner that the response was incorrect. The prescribed sequence was Incorrect -Error Consequence (p = .28). The two most likely error sequences were full physical guidance prompts (Incorrect -Physical Guidance Figure 1. Percentage treatment integrity for the final 10 blocks of 30 trials during baseline.

Treatment Integrity Reporting in Behavior Analysis in Practice 2008–2019

“…Following initial training, treatment integrity checks need to be conducted periodically to detect any procedural inaccuracies (e.g., Brand et al, 2017) or instances of treatment drift (e.g., Hansford et al, 2010) so that interventionists can quickly be retrained to conduct procedures consistent with their prescribed protocols. The staff training literature contains several examples of indirect (e.g., permanent product; e.g., Noell et al, 2005) and direct methods (e.g., checklists, data sheets; e.g., Clayton & Headley, 2019) that can be used to conduct quick treatment integrity spot checks (e.g., DiGennaro Reed et al, 2018;McIntyre et al, 2007) throughout the course of treatment.…”

“…Research is needed to assess the extent to which these features affect the risk of treatment inaccuracies. For example, descriptive studies (e.g., Brand et al, 2017Carroll et al, 2013;Cook et al, 2015) can be conducted to assess the frequency and likelihood of treatment inaccuracies when implementing procedures of varying complexity (McIntyre et al, 2007). Additionally, monitoring and reporting treatment integrity were treated as equivalent when coding studies for risk.…”

Effectiveness of rational emotive occupational health coaching in reducing burnout symptoms among teachers of children with autism

“…Specifically, the integrity checkers or raters were designated to ensure that the therapists followed the guidelines and steps embedded in the treatment manual. The checkers were developed to evaluate two major dimensions of treatment integrity which include adherence to the coaching manual, and the coaches competence 44 The adherence to REOHC manual dimension was checked using an adapted form of the Cognitive Behavioural Therapy-Adherence Scale (CBT-AS) used in earlier studies [44][45][46] The adapted checker is named which was named REOHC Adherence and is measured on a 3-point scale of not adherent = 0; partly adherent = 1, and adherent = 2. Scale and showed inter-rater high reliability (IRR = .72).…”

A parametric analysis of procedural fidelity errors following mastery of a task: A translational study

“…Future applied research could involve conducting direct observations of procedures administered for tasks that have already been mastered in the natural environment and documenting the types of errors that are likely to occur during this phase of learning (e.g., Brand et al, 2017; Brand et al, 2018; Breeman et al, 2020; Carroll et al, 2013; Cook et al, 2015; Donnelly & Karsten, 2017; Foreman et al, 2020; Kodak et al, 2018). Documenting these naturalistic error types will allow researchers to investigate how commonly occurring errors affect learning postmastery (i.e., increasing the ecological validity of the research; Brand et al, 2019).…”

“…Descriptive studies have found that procedural errors occur readily in applied settings (e.g., Brand et al, 2017, 2018; Breeman et al, 2020; Carroll et al, 2013; Cook et al, 2015; Donnelly & Karsten, 2017; Foreman et al, 2020; Kodak et al, 2018). These studies also indicate that those implementing behavioral interventions are likely to make errors when administering consequences following learners' target responses.…”