Examining Test Items for Differential Distractor Functioning Among Students With Learning Disabilities

Abstract

This study examined whether distractor choices functioned differently for students without learning disabilities than they functioned for students with learning disabilities who received no accommodation, students with learning disabilities who received a read-aloud accommodation, and students with learning disabilities who received some form of accommodation other than read-aloud. The study's purpose was twofold: (a) to examine the results of the DDF analysis to determine whether the distractors functioned differently for the various groups of students and (b) to aid in determining whether the test may be modified for those students with learning disabilities by removing a distractor while maintaining adequate test validity and information.

Psychometric Considerations for Alternate Assessments Based on Modified Academic Achievement Standards

“…Also, selection of a distractor should be negatively correlated with the total score (associated with lower total scores), where the point-biserial correlation should be less than -.20. In addition, IRT can provide even stronger evidence of distractor functioning, through a differential item functioning model for distractors, addressing differential distractor functioning across important subgroups (Green, Crone, & Folk, 1989;Middleton & Cahalan Laitusis, 2007). Qualitative evidence is equally useful when examining distractors, including evidence of cognitive aspects of distractors through think-aloud methodologies: Are the distractors eliciting the kinds of cognition we anticipated and value?…”

The Impact of Setting Accommodation on Large-scale Assessment of English Language Learners with and without Learning Disabilities: Balanced vs. Unbalanced Data in Latent Class Analyses

“…In addition, this study further examined a critical measurement issue related to unbalanced sample sizes of accommodated and comparison groups. It has been reported that there has not been a large enough sample size of students with LD in accommodation-related studies (Middleton & Laitusis, 2007). The present study scrutinized the complexity of student characteristics (LD and ELL), examinees' number sense skill, and sample size of groups of interest in relation to whether or not examinees received setting accommodation for math.…”

“…Previous studies using latent class analysis (LCA) employed data with unbalanced sample sizes (e.g., Hickendorff, Heiser, Van Putten, & Verhelst, 2009;Lesaux & Kieffer, 2010;Penuel, Boscardin, Masyn, & Crawford, 2007). However, it is not unusual for the sample size of the studied groups to be reduced through random sampling procedures at the stage of data collection, preparation and management (e.g., Cook, Eignor, Sawaki, Steinberg, & Cline, 2010;Middleton & Laitusis, 2007). For a large-scale study by Rohde et al (2001), 64 schools were randomly selected from 246 state schools where the analyzed sample of approximately 1,000 students was tested.…”

Effects of linguistic complexity and accommodations on estimates of ability for students with learning disabilities

“…With a unique focus on SLD, the purpose of this project was to evaluate the effects of accommodations and test item LC on item scores on the 4th grade National Assessment of Educational Progress (NAEP). This work builds upon research that has investigated factors that influence NAEP item scores for children with disabilities as a whole and English Language Learner (ELL students) students more specifically (e.g., Abedi, Leon, & Kao, 2008a, 2008bMiddleton & Laitusis, 2007;Stone, 2009). This previous research found that high levels of LC, or language demand, can serve as a barrier for students with disabilities and ELL students on standardized assessments, including the NAEP.…”

“…A fair accommodation must thus in some way "speak to the nature of the disability," addressing the barriers created by the interaction between the student's disability and the test item format (Fuchs, Fuchs, & Capizzi, 2005, p. 5). A valid interpretation of the accommodated score must therefore account for both the characteristics of the test and the test taker (Abedi et al, 2008a(Abedi et al, , 2008bMiddleton & Laitusis, 2007;Stone, 2009). …”