English learners who are blind or visually impaired: A participatory design approach to enhancing fairness and validity for language testing accommodations

Abstract

Standardizing accessible test design and development to meet students’ individual access needs is a complex task. The following study provides one approach to accessible test design and development using participatory design methods with school community members. Participatory research provides opportunities to empower collaborators by co-creating knowledge that is useful for assessment development. In this study, teachers of students who are visually impaired, students who are blind or are visually impaired, English language teachers, and test administrators provided feedback at critical stages of the development process to explore the construct validity of English language proficiency (ELP) assessments. Students who are blind or visually impared need to be able to show what they know and can do without impact from construct-irrelevant variance like language acquisition or disability characteristics. Building on our iterative accessible test design, development, and delivery practices, and as part of a large project on English-learner proficiency test accessibility and usability, we collected rich observation and interview data from 17 students who were blind or visually impaired and were enrolled in grades kindergarten through Grade 12. We examined the ratings and item metadata, including assistive technology preferences and interactions, while we used grounded theory approaches to examine qualitative thematic findings. Implications for research and practice are discussed.

Language assessment accommodations: Issues and challenges for the future

“…Several papers reference the concept of universal design as the preferred theoretical foundation for language test design and development (Christensen et al, 2023;Kim et al, 2023;Guzman-Orth et al, 2023). This approach, originally derived from the field of architecture in the United States (Case, 2008), proposes a set of principles whereby assessments are intentionally and proactively designed from the earliest stage of construction to be maximally accessible to all users, regardless of any special needs they may have (cf., the planning and design of public buildings as disabled-friendly).…”

“…This makes classroom teachers key assessment stakeholders, who can help to bridge teaching, learning, and assessment contexts for all students but especially those with special needs. Happily, several of the special issue papers draw directly on insights from the teachers of students requiring accommodations, emphasizing the importance of their input at varying stages of test design and development, as well as in interpreting test outcomes (Guzman-Orth et al, 2023;Motteram et al, 2023).…”

“…It is good to see far more evidence nowadays of test providers and researchers engaging collaboratively with multiple stakeholder groups who have insights to share in this endeavour, including lived experience. The six papers in this special issue have offered an extensive list of test stakeholders who can or should be consulted, including: English language teachers (Guzman-Orth et al, 2023;Motteram et al, 2023); specialist teachers for learners with disabilities (Guzman-Orth et al, 2023); school leaders and administrators (Motteram et al); accessibility experts (Christensen et al); parents (Motteram et al); test invigilators and examiners (Guzman-Orth et al 2023;Motteram et al, 2023); and, perhaps most importantly, test takers themselves (Kim et al, 2023;Guzman-Orth et al, 2023). Guzman-Orth et al (2023), in particular, stressed the benefits of a participatory design approach that empowers collaborators and helps to co-create knowledge and understanding that are essential to successful test development.…”