Treatment Sensitivity of Direct Behavior Rating–Multi-Item Scales in the Context of a Daily Report Card Intervention

Abstract

The primary goal of this study was to assess the treatment sensitivity of four newly developed Direct Behavior Rating–Multi-Item Scales (DBR-MIS) that assess the domains of academic engagement, disruptive behavior, organizational skills, and oppositional behavior in the context of a Daily Report Card (DRC) intervention. To achieve this goal, we first evaluated the integrity and effectiveness of the DRC intervention in this sample. Participants included six elementary school teachers, each of whom delivered a DRC intervention with one student from their classroom, while completing DBR-MIS ratings on a daily basis for 2 months. Results confirmed the effectiveness of the DRC intervention (all DRC target behaviors demonstrated improvement, with at least half demonstrating improvement that was moderate to large in magnitude) and revealed a positive relationship between DRC implementation integrity and student outcomes. We found strong evidence for the treatment sensitivity of the DBR-MIS assessing academic engagement, disruptive behavior, and organizational skills. Results for the treatment sensitivity of the DBR-MIS oppositional scale were inconclusive. Implications for progress monitoring using the recently developed DBR-MIS are discussed.

Use of Mixed Methods in Emotional/Behavioral Disorders Intervention Research

“…Single-case design was defined as a research study in which the dependent variable was repeatedly measured during a baseline condition, followed by an intervention condition with changes in level, trend, and variability that are used to assess the presence of experimental control (Gast & Ledford, 2014). Examples of single-case designs included withdrawal and reversal design, multiple baseline design, multiple probe design, multitreatment design, alternating treatment design, multi-element design, adapted alternating treatments design, parallel treatments design, changing criterion design, concurrent operant design, repeated acquisition design, brief experimental design, and combination designs (e.g., Hustus et al, 2020).…”

Implementation resources to support teachers’ use of behavioral classroom interventions: protocol of a randomized pilot trial

“…For this study, teachers will rate behaviors in two domains: Engagement (5 items rating frequency of these behaviors on a 7-point scale ranging from Never to Almost Always) and Disruptive Behavior (5 items rating degree to which each item is a problem on a 7-point scale ranging from Not a Problem to Serious Problem). These domains of the DBR-MIS have demonstrated treatment sensitivity [ 17 ]. Teachers will complete these ratings at baseline, midpoint and endpoint.…”

Exploring teacher adoption, adaptation, and implementation of a daily report card intervention when using the daily report card online platform

“…To screen students and to monitor progress over time, teachers completed multi-item Daily Behavior Ratings on the DRCO platform (DBR) (Hustus et al, 2020;Matta et al, 2020). Two DBRs assess positive behaviors (Organizational Skills and Engagement; αs > 0.81 across all time points) and two DBRs assess problem behaviors (Opposition and Disruptiveness αs > 0.85 across all time points).…”

“…This measure was completed prior to the start of the DRC intervention, monthly during intervention, and in the spring. Studies document the treatment sensitivity of these measures (Hustus et al, 2020;Matta et al, 2020) and that they are capable of identifying children with behavior problems (Daniels et al, 2020). DBR scores are used in Aim 4.…”