Accuracy of Readability Estimates' Predictions of CBM Performance.

Abstract

Researchers investigating the psychometric properties of curriculum-based measures in reading (R-CBM) have typically used readability formulas to create passages of similar difficulty. Despite their efforts, many researchers still find excessive, uncontrolled systematic error (presumably due to passage difficulty) in R-CBM data. This study investigates the validity of eight readability formulas by examining their ability to predict students' words read correctly in a minute (WRCM). The study is based on the premise that as grade levels assigned to passages by readability formulas increase, students' WRCM should decrease. Results indicate a modest relationship between reading fluency and passage difficulty as indicated by the.eight readability formulas. The formulas most commonly employed in R-CBM research were the poorest predictors. Findings suggest that efforts beyond the use of currently published readability estimates are needed in order to procure equivalent forms for R-CBM monitoring.Curriculum-based measurement in reading (R-CBM) is an assessment procedure in which students read passages and the number of words read correctly in one minute (WRCM) is recorded. R-CBM was placed on the "approved" list by the Reading First Assessment Committee (Kame'enui, 2002) meaning that it meets or exceeds standards of psychometric accuracy. R-CBM is an invaluable tool for many practitioners because it can assist in answering two common questions: (1) How well is this child reading? and (2) Is he or she making progress in reading?

Statistical equating with measures of oral reading fluency

“…Conversely, with more rigorous purposive selection employed to identify passages of a single difficulty, error variance is expected to diminish. Poncy et al (2005) confirmed this principle using generalizability theory, and Ardoin et al (2005) highlighted it while revealing the inadequacy and unreliability of readability measures. CBM-R publishers go to great lengths to write and choose forms of similar difficulty (e.g., Good & Kaminski, 2002;Howe & Shinn, 2002).…”

“…In order to maintain simplicity, as CBM-R was geared toward classroom teachers, this process was considered sufficient for approximating equivalent forms (e.g., Deno, 1985;Deno, Marston, & Tindal, 1986). With their extended use and adaptation, parallelism of passages is now typically improved by (a) increasing the number of passages to choose from, (b) ensuring that passages converge on a specific difficulty level (Hintze & Christ, 2004;Poncy et al, 2005), and (c) using the appropriate screening or selection criteria throughout the passage selection process (Ardoin et al, 2005). Convergence on a single difficulty level is critical to identifying potential reading passages.…”

“…However, even with such high correlations and identical readability scores, parallel forms have not been reached, as neither of these indices takes into account absolute score differences across forms. Ardoin et al (2005) referred to the unreasonableness of expecting a set of passages to ever be exactly equivalent. After discussing the poor performance of readability formulas they concluded: "It may be that the best and only valid measure of the difficulty of a passage when evaluating fluency is the actual reading fluency of a group of same-age students" (p. 18).…”

Form effects on the estimation of students' oral reading fluency using DIBELS

“…Ardoin, Suldo, Witt, Aldrich, & McDonald (2005) investigated the validity of eight reliability formulas as predictors of student's WCPM. The stories were from a reading program used in both third and fourth grade.…”

The response in response to intervention: Evaluating the utility of assessing maintenance of intervention effects

“…For example, Hintze and Christ (2004) recommended the use of readability formulas, and Shinn (2002) in Best Practices in School Psychology IV recommended that a minimum of 10 R-CBM data points be collected to account for measurement error. More recent studies have suggested that readability formulas fail to adequately control passage difficulty (Ardoin, Suldo, Witt, Aldrich, & McDonald, 2005) and that biweekly data collected across 10 weeks rather than only 5 weeks is needed (Christ, 2006, April). The additional data points ideally result in smaller SEbs (Christ, 2006, April) and also may provide time for intervention effects to generalize to nonintervention materials.…”