Improving Mathematics Screening in Middle School

Abstract

Common universal screening methods for determining math risk in middle school grades may not result in optimal diagnostic accuracy. We evaluated current screening practices and several potential modifications for predicting math proficiency on an end-of-year state test in a suburban school district. Previously, the district used the Measures of Academic Progress (MAP) to determine student risk in fall. Creating local cut scores for the preceding-year state test scores and the MAP resulted in the most accurate and efficient methods of assessing risk. Multiskill computation and application curriculum-based measures added little relative value to the combination of MAP and preceding-year state test scores. Results suggest schools could improve their screening practices by first evaluating their current procedures and, if unacceptable, determining whether local cut scores provide the desired improvements to accuracy before considering the adoption of additional measures.

Evaluating aimswebPlus math as a universal screening measure in upper elementary and middle school.

“…The recent changes made to popular math CBM tools suggest that the previous findings regarding the diagnostic accuracy of math CBM tools may have limited relevance. More specifically, the inclusion of composite scores to represent math proficiency in computation and applications, alignment with Common Core State Standards, and updated thresholds for determining risk may result in diagnostic accuracy estimates that are similar to those found for computer-adaptive tests or for prior year statewide achievement tests (Klingbeil et al, 2019; Van Norman et al, 2017; VanDerHeyden et al, 2017). When predicting year-end proficiency in Grades 3 through 8, the vendor-reported AUC values were good to excellent (Compton et al, 2006) for the Acadience Math composite scores (Gray et al, 2019) and aimswebPlus composite scores (NCS Pearson Inc., 2017).…”

“…Across several studies, computation and application CBM scores have been associated with adequate to good AUC values in upper elementary (Keller-Margulis et al, 2008; VanDerHeyden et al, 2017) and middle school grades (Klingbeil et al, 2019; Strait et al, 2018). AUC values for application CBM probes and computation CBM probes are generally similar with small differences in diagnostic accuracy related to the grade and season in which the measures are administered.…”

“…AUC values for application CBM probes and computation CBM probes are generally similar with small differences in diagnostic accuracy related to the grade and season in which the measures are administered. Despite AUC values in the adequate to good range, when researchers have derived cut-scores that result in highly sensitive decisions, computation and application CBMs often result in decisions with poor specificity (e.g., Klingbeil et al, 2019; Leblanc et al, 2012).…”

“…There is some evidence to suggest that screening decisions made based on prior year statewide achievement test scores may be more sensitive than decisions made based on computation or applications CBMs (Klingbeil et al, 2019; VanDerHeyden et al, 2017). Results from the few studies that directly compared the diagnostic accuracy of math CBMs to prior year state tests are corroborated by other research documenting excellent AUC values for the prior year state tests (Nelson et al, 2016).…”

“…As with computation and application CBM measures, there is evidence that deriving local cut-scores for prior year statewide achievement tests can improve sensitivity to desirable levels. However, unlike computation or application CBM scores, local cut-scores designed to maximize sensitivity appear to maintain adequate specificity (Klingbeil et al, 2019; Nelson et al, 2017; Van Norman et al, 2017).…”