Comparison of Predictive Validity and Diagnostic Accuracy of Screening Measures of Reading Skills

Abstract

Assessment data must be valid for the purpose for which educators use them. Establishing evidence of validity is an ongoing process that must be shared by test developers and test users. This study examined the predictive validity and the diagnostic accuracy of universal screening measures in reading. Scores on three different universal screening tools were compared for nearly 500 second-and third-grade students attending four public schools in a large urban district. Hierarchical regression and receiver operating characteristic curves were used to examine the criterion-related validity and diagnostic accuracy of students' oral reading fluency (ORF), Fountas and Pinnell Benchmark Assessment System (BAS) scores, and fall scores from the Measures of Academic Progress for reading (MAP). Results indicated that a combination of all three measures accounted for 65% of the variance in spring MAP scores, whereas a reduced model of ORF and MAP scores predicted 60%. ORF and BAS scores did not meet standards for diagnostic accuracy. Combining the measures improved diagnostic accuracy, depending on how criterion scores were calculated. Implications for practice and future research are discussed. C 2015 Wiley Periodicals, Inc.

Evaluating the Criterion Validity and Classification Accuracy of Universal Screening Measures in Reading

“…Generally, the findings from the current study were similar to or higher than the range of correlations reported in previous studies for predictive validity using either MAP or state assessments as the criterion measure (Ball & O'Connor, 2016;Klingbeil et al, 2015;. Additionally, the recent study conducted by MAP test publishers corresponded precisely with the concurrent validity findings from the first cohort in the present study (NWEA, 2016).…”

“…Strong associations between MAP and the state assessment corroborate previous research of strong associations between MAP and other standardized assessments of reading achievement (Ball & O'Connor, 2016;January & Ardoin, 2015;Klingbeil et al, 2015Klingbeil et al, , 2017. However, the concurrent relation between MAP and the standardized assessment was slightly lower than findings from the one previous study that evaluated concurrent validity with a norm-referenced assessment (January & Ardoin, 2015).…”

“…In a study with second-and third-grade students, Klingbeil, McComas, Burns, and Helman (2015) examined the criterion-related validity and classification accuracy of fall MAP scores in predicting end-of-year reading proficiency using spring MAP scores.…”

Development of a word reading test: Identifying students at-risk for reading problems

“…This misclassification has costs: false negatives will not receive the support they need and the false positives will be assigned to unneeded educational support, wasting human and financial resources necessary for intervention with students with reading disabilities (Compton, Fuchs, Fuchs, & Bryant, 2006;Stevens, 1992). Due to the consequences of flagging false negatives (Klingbeil, McComas, Burns, & Helman, 2015;Slocum, 2002), most of the test developers choose to maximize sensitivity, which reduces the number of false negatives (Johnson et al, 2009). Concerning the minimally acceptable levels of sensitivity and specificity, some authors recommended 0.90 for sensitivity and 0.80 for specificity (Compton et al, 2006) whereas others suggested a value for sensitivity of 0.80 and specificity of 0.70 (Johnson et al, 2009).…”

Illuminating the Complexity of Oral Reading Fluency: A Multiple Lens Approach

“…These fluency measuring tools have been investigated over the years, as have their implementation into educational settings (Ardoin et al, 2013; Chard et al, 2002; Good & Kaminski, 2002; Klingbeil et al, 2015; Valencia et al, 2010; Wise et al, 2010). In practice, reading fluency assessment and fluency “enhancing” instruction draw from behaviorist component skills approaches (Damico & Nelson, 2010; Klingbeil et al, 2015; National Institute of Child Health and Human Development, 2000). In this context, reading fluency is typically referred to as accurate and automatic decoding of the words in the text (LaBerge & Samuels, 1974), accompanied by expressive interpretation of the text to achieve optimal comprehension (Rasinski, 2004).…”