Technical Adequacy of Response to Intervention Decisions

Abstract

Perhaps the greatest value of response to intervention (RTI) as a decision framework is that it brings attention to variables (e.g., mastery of prerequisite skills, frequency of instructional corrective feedback, reinforcement schedules for correct responding) that if changed might make a meaningful difference for students (e.g., child rate of learning is accelerated and child learns to read). Yet, RTI is also a model that has diagnostic implications. Classification agreement analyses can quantify accuracy and efficiency of decisions within RTI systems and can indicate which procedures and decision rules lead to the best decisions. Because RTI systems are dynamic and procedures vary between schools and implementers, classification agreement analyses, when not misapplied or misinterpreted, are ideal for quantifying utility of various decision models. This article offers a critical analysis of sensitivity, specificity, positive predictive power, and negative predictive power estimates and discusses the development of sophisticated models of decision making.

Evaluating Procedures for Reducing Measurement Error in Math Curriculum-Based Measurement Probes

“…M-CBM has been used to facilitate decisions about student progress (Shapiro et al, 2005;VanDerHeyden & Burns, 2005), and it is widely recognized that using scores that are susceptible to measurement error can lead to inaccurate screening and progress decisions (Fuchs, 2003;VanDerHeyden, 2011). The importance of using measurement instruments with the strongest possible psychometric properties is widely recognized as a professional standard (AERA, APA, & NCME, 1999;Betts, 2010).…”

Progress Monitoring With Computer Adaptive Assessments: The Impact of Data Collection Schedule on Growth Estimates

“…Data‐based decisions are often made at the student level; however, many educators and scholars are also interested in the degree to which larger groups of students respond to instruction or intervention across time (Bryant, Bryant, Gersten, Scammacca, & Chavez, ; Shinn, ). Guidelines exist for the evaluation of progress‐monitoring measures (e.g., Glover & Albers, ) and to some extent, for the use of those measures in practice (VanDerHeyden, ; Vaughn & Fletcher, ); however, there are some discrepancies in what constitutes best practice in the application of screening and progress‐monitoring measures. For example, although more frequent progress‐monitoring data might yield more precise growth estimates, the recommended number of observations varies (Jenkins, Graff, & Miglioretti, ; Mercer & Keller‐Margulis, ) as does the number of observations that actually occur in practice (Mellard, McKnight, & Woods, ; Prewett et al., ).…”

Classification agreement analysis of Cross-Battery Assessment in the identification of specific learning disorders in children and youth

“…Classification agreement analyses have been used for many years in the field of medicine to examine the usefulness of diagnostic tests for various medical conditions (see Youngstrom, 2013). These methods have been employed in recent years in education to examine the accuracy of screening measures for early identification of students who are at risk for educational failure (e.g., VanDerHeyden, 2010VanDerHeyden, , 2011.…”