Addressing the Large Standard Error of Traditional CBM-R: Estimating the Conditional Standard Error of a Model-Based Estimate of CBM-R

Abstract

<jats:p> Curriculum-based measurement of oral reading fluency (CBM-R) is widely used across the country as a quick measure of reading proficiency that also serves as a good predictor of comprehension and overall reading achievement, but has several practical and technical inadequacies, including a large standard error of measurement (SEM). Reducing the SEM of CBM-R scores has positive implications for educators using these measures to screen or monitor student growth. The purpose of this study was to compare the SEM of traditional CBM-R words correct per minute (WCPM) fluency scores and the conditional SEM (CSEM) of model-based WCPM estimates, particularly for students with or at risk of poor reading outcomes. We found (a) the average CSEM for the model-based WCPM estimates was substantially smaller than the reported SEMs of traditional CBM-R systems, especially for scores at/below the 25th percentile, and (b) a large proportion (84%) of sample scores, and an even larger proportion of scores at/below the 25th percentile (about 99%) had a smaller CSEM than the reported SEMs of traditional CBM-R systems. </jats:p>

Comparing the Growth and Predictive Performance of a Traditional Oral Reading Fluency Measure With an Experimental Novel Measure

“…In addition, the correlation between the latent intercept and slope factors for the CORE models were negative and moderate in magnitude, but were positive and small to moderate in magnitude for the traditional CBM-R models. These results may reflect of a ceiling effect, but that is not supported by the data; rather, these results suggest the model-based CORE scores are more sensitive to growth for students at risk of poor reading outcomes (i.e., lower fall WCPM scores), a finding that is supported by previous research that found increased precision (i.e., smaller conditional standard error of measurement) for CBM-R scores at/below the 25th percentile (Nese & Kamata, 2020a). This finding should be further examined by future research.…”

“…CORE reshapes oral reading fluency and traditional CBM-R assessment by allowing group administration, more than one minute of reading, multiple passages, machine scoring, and WCPM scale scores. The benefits include reduced human administration cost and errors (Nese & Kamata, 2020b), and reduced standard error of measurement (Nese & Kamata, 2020a). The results of this study suggest increased measurement precision for the model-based CORE scores compared to traditional CBM-R, providing preliminary evidence that CORE can be used for consequential assessment.…”

“…Most important, to address passage inequivalence and to improve score reliability and precision, CORE developed and validated shorter passages (Nese & Kamata, 2020b), which were equated, horizontally scaled and vertically linked with an alternative scale metric based on a latent-variable psychometric model of speed and accuracy (Kara et al, 2020). These contributions resulted in substantially smaller standard error of measurement for the model-based CORE scores compared with Traditional CBM-R scores, especially for students at risk of poor reading outcomes, providing CBM-R scores that are sensitive to instructional change (Nese & Kamata, 2020a).…”

Evidence for automated scoring and shorter passages of CBM-R in early elementary school.

“…Nevertheless, research has shown that the use of shorter passages, in combination with advanced psychometric modeling, has the potential to reduce the standard error and increase the reliability of WCPM scores (Kara et al, 2020). A smaller standard error for CBM-R scores would provide educators with more accurate scores with which to monitor student CBM-R progress over time (Nese & Kamata, 2020), and thus increase the consequential validity of the decisions based on those scores.…”

“…Traditional CBM-R scores are generally associated with large measurement error, or large SE (Ardoin & Christ, 2009; Christ & Silberglitt, 2007; Christ et al, 2012; Poncy et al, 2005), which can affect the interpretations and consequences of the assessment results as teachers use CBM-R fluency scores to screen for students at risk of poor reading outcomes and monitor student progress to inform instruction (Christ & Coolong-Chaffin, 2007). Along with a psychometric modeling approach (Kara, Kamata, Potgieter, & Nese, 2020; Potgieter, Kamata, & Kara, 2017), having students read several shorter passages may be part of a larger solution to reduce the large SE associated with traditional CBM-R fluency scores and increase score reliability (Nese & Kamata, 2020). Thus, in this study we explore whether shorter passages (25 to 85 words) read in their entirety function comparably to traditional CBM-R passages (about 250 words) read for 60 s.…”