Generalizability Theory Applied to Reading Assessments for Students With Significant Cognitive Disabilities

Tindal, Gerald; Yovanoff, Paul; Geller, Josh

doi:10.1177/0022466908323008

Cited by 11 publications

(12 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In addition, implementation of these supports has been shown to not compromise the reliability of the test. In a study by Tindal, Yovanoff, and Gellar (2010), generalizability theory analyses suggested that the test administrator was a negligible facet in the reliability of the test. These results and the Tindal et al results suggest that well-defined administrator-provided supports in standardized, performance-based alternate assessments can help students access the test while not compromising reliability and the validity of test-based inferences.…”

Section: Discussionmentioning

confidence: 99%

Test Design Considerations for Students With Significant Cognitive Disabilities

2013

Self Cite

View full text Add to dashboard Cite

Students with significant cognitive disabilities present an assessment dilemma that centers on access and validity in largescale testing programs. Typically, access is improved by eliminating construct-irrelevant barriers, while validity is improved, in part, through test standardization. In this article, one state's alternate assessment data were analyzed to determine the impact of (a) administration supports based on students' level of independence and (b) a scaffold test administration format. Using structural equation modeling, we tested the extent to which students' level of independence mediated the relation between disabilities and latent content knowledge scores. We then tested the invariance of the measurement model across administration formats. The results provide evidence that these supports help students access the test without compromising the validity of test-based inferences.

show abstract

Section: Discussionmentioning

confidence: 99%

Test Design Considerations for Students With Significant Cognitive Disabilities

2013

Self Cite

View full text Add to dashboard Cite

show abstract

“…While perfect agreement and kappa analyses have been used to measure rater agreement and reliability for classroom and teacher observations, generalizability and decision studies are able to account for variability that traditional interrater agreement analyses cannot (Brennan, 2001; Cronbach et al, 1972; Hill, Charalambous, Blazar, et al, 2012; Tindal, Yovanoff, & Geller, 2010). The use of generalizability theory, or G theory, to inform the use of the RESET tool is important because it accounts for multiple sources of variance that cannot be measured in interrater agreement analyses like kappa, and reflects a much more realistic measurement of what happens in a classroom.…”

Section: Generalizability Theorymentioning

confidence: 99%

Measuring Rater Reliability on a Special Education Observation Tool

Semmelroth

Johnson

2013

Assessment for Effective Intervention

View full text Add to dashboard Cite

This study used generalizability theory to measure reliability on the Recognizing Effective Special Education Teachers (RESET) observation tool designed to evaluate special education teacher effectiveness. At the time of this study, the RESET tool included three evidence-based instructional practices (direct, explicit instruction; whole-group instruction; and discrete trial teaching) as the basis for special education teacher evaluation. Five raters participated in two sessions to evaluate special education classroom instruction collected from two school years, via the Teachscape 360-degree video system. Data collected from raters were analyzed in a two-facet “partially” nested design where occasions (o) were nested within teachers (t), o:t, and crossed with raters (r), {o:t} x r. Results from this study are in alignment with similar studies that found multiple observations and multiple raters are critical for ensuring acceptable levels of measurement score reliability. Considerations for the feasibility of practice should be observed in future reliability and validity studies on the RESET tool, and further work is needed to address the lack of research on rater reliability issues within special education teacher evaluation.

show abstract

“…Additional analyses were completed to test the hypothesis of administration format on measurement reliability. Using generalizability theory (Brennan, 2000; Yovanoff, Tindal, & Geller, 2010), reliability was estimated by computing the systematic variation due to students, format, and occasions, and the interaction of these sources of variability. While student true score variability is desirable, other sources of variability are regarded as systematic and random “error” (attributable to sources such as occasions, format, student).…”

Section: Methodsmentioning

confidence: 99%

Alternate Assessment Formats for Progress Monitoring Students With Intellectual Disabilities and Below Average IQ: An Exploratory Study

Jones

Gifford

Yovanoff

et al. 2018

Focus Autism Other Dev Disabl

Self Cite

View full text Add to dashboard Cite

As part of standards-based reforms, there is increasing emphasis on ensuring that students with moderate intellectual disabilities (ID), including students with Autism Spectrum Disorders (ASD), learn to read. There is also converging evidence that explicit teaching of letter sounds, phonics, and sight words is effective for this population, but that students’ responsiveness varies. A critical part of individualizing reading instruction for students with disabilities is the reliable assessment of progress and mastery of reading skills. However, assessment of many students with ID and students with ASD is challenging because of attention, behavioral, and communication issues related to testing situation; therefore, obtaining consistent results often proves to be a difficult task. We hypothesized that alternate assessment presentation formats, as a testing accommodation, would improve the reliability, validity, and consistency of assessment performance. In this study, three different presentation formats—word lists, flash cards, and PowerPoint presentation—were used when administering proximal, curriculum-based reading assessments to determine whether a particular format increased student engagement, reduced the need for prompts, and increased accuracy of identifying known items on the test. While statistical analyses did not support the hypothesis of a format by student effect, visual analysis of the data did suggest that the number of prompts required varied by student as a function of assessment format. Most noteworthy, assessment reliability, estimated with generalizability theory, indicated that reliability increased as a function of format by student.

show abstract

Generalizability Theory Applied to Reading Assessments for Students With Significant Cognitive Disabilities

Cited by 11 publications

References 29 publications

Test Design Considerations for Students With Significant Cognitive Disabilities

Test Design Considerations for Students With Significant Cognitive Disabilities

Measuring Rater Reliability on a Special Education Observation Tool

Alternate Assessment Formats for Progress Monitoring Students With Intellectual Disabilities and Below Average IQ: An Exploratory Study

Contact Info

Product

Resources

About