Examining Potential Bias in Screening Measures for Middle School Students by Special Education and Low Socioeconomic Status Subgroups

Abstract

To provide timely and effective supports for students reading below grade level, schools require methods for quickly and accurately identifying those students in need. One method for identifying those students is through universal screening. Assessments such as oral reading fluency (ORF) and Maze reading comprehension are commonly used as screening assessments in middle grades. The current study examined ORF and Maze for evidence of bias across two subgroups known to be at increased risk for failure in reading: (a) students with learning disabilities and (b) students from low‐income households. Data from 4,215 students in the sixth (n = 1,126), seventh (n = 1,361), and eighth grades (n = 1,728) were analyzed. Results indicate no significant differences in predictive validity for students from low‐income households compared to students from middle and upper income households. For students with learning disabilities only 8th grade scores showed any evidence of bias compared to students without diagnosed disabilities. Implications and recommendations for future research are discussed.

Using Curriculum-Based Measurement to Predict Student Performance on an Eighth-Grade State Reading Assessment

“…As in previous research conducted in different states (Espin et al, 2010; Shin, 2017; Silberglitt et al, 2006; Ticha et al, 2009; Yeo, 2010), scores on the fall and spring CBMs shared strong positive unique relationships with scores on a statewide reading assessment. In contrast to some previous findings, however (Shin, 2017; Stevenson et al, 2016), CBM validity did not vary by student characteristics. These findings, if replicated, suggest that the benefit of more frequent measurement of reading skills afforded by CBMs may not be limited to the primary grades.…”

“…In another recent study, meal and disability status were examined as moderators of the relationship between ORF and maze on state test performance. Findings indicated that ORF was a better predictor of state test performance for eighth-grade students without disabilities than for students with disabilities (Stevenson et al, 2016).…”

“…Since this meta-analysis was completed, a handful of additional studies have examined the predictive validity of ORF CBM in the middle grades, and reported results have been mixed (Ford, Missall, Hosp, & Kuhle, 2017; Muijselaar, Kendeou, de Jong, & van den Broek, 2017; Stevenson, 2017; Stevenson, Reed, & Tighe, 2016). One of the studies revealed that when ORF CBM scores were correlated with performance on a state test, the validity coefficient obtained in sixth grade was .59 while the same relationship in a sample of eighth-grade students ( r = .31) was notably lower (Stevenson et al, 2016). Another study using a norm-referenced commercial test as the criterion measure also reported the correlations weakened by grade level (Muijselaar et al, 2017).…”

“…Since publication of Shin’s meta-analysis, the predictive validity of maze CBM in the middle grades has remained an interest of several researchers (Chung, Espin, & Stevenson, 2018; Conoyer et al, 2017; Ford, Missall, Hosp, & Kuhle, 2016; Muijselaar et al, 2017; Stevenson, 2017; Stevenson et al, 2016). As with ORF, Stevenson et al (2016) reported that the magnitude of maze CBM predictive validity coefficients weakened from .54 to .15 over the middle grades, a more dramatic deterioration than was observed in their ORF CBM data. Moreover, Muijselaar et al (2017) also found that while maze correlations with a norm-referenced criterion weakened somewhat from fourth to ninth grade, they remained above .69.…”

Universal Screening Methods and Models: Diagnostic Accuracy of Reading Assessments

“…There are limited research studies that analyzed multiassessments to identify approaches (Compton, Fuchs, Fuchs, & Bryant, 2006; O’Connor & Jenkins, 1999), but they tended to focus on screening of early elementary school–aged students. Several studies were found that examined the screening accuracy of tools for upper elementary school students on a high-stakes assessment (Baker et al, 2015; Decker, Hixon, Shaw, & Johnson, 2014; Denton et al, 2011; McGlinchey & Hixon, 2004; Nese, Park, Alonzo, & Tindal, 2011; Shapiro, Solari, & Petscher, 2008; Stevenson, Reed, & Tighe, 2016). Only one study was found that explored a multigate approach (Klingbeil, Nelson, Van Norman, & Birr, 2017).…”