Design and Psychometric Evaluation of American Sign Language Translations of Usability Questionnaires

Abstract

To promote greater inclusion of people who are Deaf and Hard of Hearing (DHH) in studies conducted by Human-Computer Interaction (HCI) researchers or professionals, we have undertaken a project to formally translate several standardized usability questionnaires from English to ASL. Many deaf adults in the U.S. have lower levels of English reading literacy, but there are currently no standardized usability questionnaires available in American Sign Language (ASL) for these users. A critical concern in conducting such a translation is to ensure that the meaning of the original question items has been preserved during translation, as well as other key psychometric properties of the instrument, including internal reliability, criterion validity, and construct validity. After identifying best-practices for such a translation and evaluation project, a bilingual team of domain experts (including native ASL signers who are members of the Deaf community) translated the System Usability Scale (SUS) and Net Promoter Score (NPS) instruments into ASL and then conducted back-translation evaluations to assess the faithfulness of the translation. The new ASL instruments were employed in usability tests with DHH participants, to assemble a dataset of response scores, in support of the psychometric validation. We are disseminating these translated instruments, as well as collected response values from DHH participants, to encourage greater participation in HCI studies among DHH users. CCS Concepts: • Human-centered computing → Empirical studies in HCI; Accessibility design and evaluation methods;

Towards Inclusive Software: Accessible Questionnaires for Deaf People

“…In relation to the questionnaire technique, Berke et al [Berke et al 2019] investigated how to translate standardized usability questionnaires into American Sign Language. The authors stated that to ensure that technology is accessible to diverse users, researchers and designers should ideally include people with disabilities during the studies.…”

Psychometric Evaluation of the F-SUS: Creation and Validation of the French Version of the System Usability Scale

“…Finally, we note that the SUS has been transcribed into American Sign Language (Berke et al, 2017), as well as pictorial language (Baumgartner et al, 2019).…”

Mesure de la sensibilité et de la signification de la version française du System Usability Scale

“…Relevons pour finir que le SUS a été transcrit dans le langage des signes américain [6], mais aussi en langage pictural [5].…”