Applied Empiricism: Ensuring the Validity of Causal Response to Intervention Decisions

Abstract

School personnel make a variety of decisions within multitiered problem-solving frameworks, including the decision to assign a student to group-based support, to design an individualized support plan, or classify a student as eligible for special education. Each decision is founded upon a judgment regarding whether the student has responded to intervention. These and other conclusions are inherently causal, thus requiring that educators carefully consider the internal, construct, and conclusion validity of each decision to ensure its defensibility. Researchers have identified multiple variables that are likely to moderate these validities, including the integrity with which interventions are implemented, the psychometric adequacy of progress-monitoring tools, the extent to which interventions and supports are matched to a student's needs, and the approach to single-case research design. We therefore review each of these variables in the interest of assisting practitioners to design acceptable and valid multitiered frameworks of prevention and service delivery. Keywords Response to intervention . Research methods . ValiditySchool-based multitiered frameworks, including response to intervention (RTI), have gained prominence as general service delivery models, emphasizing risk minimization through prevention, and early identification and intervention (Vaughn and Fuchs 2003). Recently, Kratochwill et al. (2012) reviewed the role of science within school psychological practice, illustrating the relationships between the RTI, scientist-practitioner, and evidence-based practice movements. The authors specified two main roles filled by school psychology scientistpractitioners. The first pertains to the consumption of scientific evidence. RTI is founded upon the use of evidence-based practices; schools are required to use interventions supported by scientific evidence, which serves as a preliminary indicator that such strategies are appropriate and likely to promote student outcomes. Judgment as to whether evidence is both scientific and sufficient to support applied use is a primary function of the scientist-practitioner . The second role is the generation of scientific evidence. Training in research methods and data analysis permits the scientist-practitioner to engage in practice-based research for the purpose of collecting evidence of the effectiveness of interventions and supports (Sidman 2011). Such research calls for the use of sophisticated methods and analyses, which together support the validity of inferences regarding intervention effectiveness and increase the likelihood of the evidence's contribution to the peer-reviewed literature. Many have called for the proliferation of such research, as it stands to both enhance the dissemination of evidence-based practices and determine how practices may be adapted to fit local contexts and enhance effectiveness .Although a focus on school psychologists as generators of evidence is necessary and crucial, we see potential for expansion to the definition of the scientist...

Data-Driven Delivery of Implementation Supports in a Multi-Tiered Framework: A Pilot Study

“…Unfortunately, research results consistently demonstrate that implementers display a variety of treatment integrity patterns and many, if not most, implementers struggle to consistently deliver interventions in the absence of systematic follow‐up (Noell, Witt, Gilbertson, Ranier, & Freeland, ; Sanetti, Fallon, & Collier‐Meek, ). When interventions are delivered without sufficient treatment integrity, they are not only less effective in improving student outcomes (Fryling, Wallace, & Yassine, ), but it is inappropriate to make decisions about their impact within a multi‐tiered system of support (Kilgus et al., ; Noell & Gansle, ). These findings point to the importance of ensuring that research‐based interventions are implemented as planned; lack of treatment integrity may be the biggest hurdle in realizing the full potential of tiered delivery models.…”

“…Treatment integrity (i.e., the extent to which an intervention is delivered as planned) has gone from being largely ignored two decades ago to being regularly acknowledged as a critical aspect of education intervention research and practice (Cochrane & Laux, ; Sanetti & Kratochwill, ). Although treatment integrity is relevant for all student interventions, the increased focus on this topic has been hastened by the widespread adoption of multi‐tiered systems of support as a delivery model for academic and behavioral interventions (e.g., Response‐to‐Intervention, Positive Behavior Interventions and Supports; Kilgus, Collier‐Meek, Johnson, & Jaffery, ). In this delivery model, high‐quality curricula are delivered to all students, and regular assessment ensures that those students who are not making adequate progress receive interventions in a targeted and then individualized manner.…”

Single-case design effect-size distributions: Association with procedural parameters.

“…and How big is the behavior change? ), they are both critical when evaluating the impacts of school-based interventions (Kilgus et al, 2014).…”

Single-Case Data Analysis: A Practitioner Guide for Accurate and Reliable Decisions

“…The ability of teachers to discern qualities of evidence as it unfolds in the professional literature is also an empowering skill set (Wendt & Miller, 2012). Professionals charged with monitoring student progress must know how to select educational programs, implement them with integrity, design graphic displays, demonstrate program effects on target behaviors with reliable measures, and analyze program effects with technical accuracy (Kilgus, Collier-Meek, Johnson, & Jaffery, 2014; Lane & Gast, 2014).…”