Toward fairness in AI for people with disabilities SBG@a research roadmap

Abstract

AI technologies have the potential to dramatically impact the lives of people with disabilities (PWD). Indeed, improving the lives of PWD is a motivator for many state-of-the-art AI systems, such as automated speech recognition tools that can caption videos for people who are deaf and hard of hearing, or language prediction algorithms that can augment communication for people with speech or cognitive disabilities. However, widely deployed AI systems may not work properly for PWD, or worse, may actively discriminate against them. These considerations regarding fairness in AI for PWD have thus far received little attention. In this position paper, we identify potential areas of concern regarding how several AI technology categories may impact particular disability constituencies if care is not taken in their design, development, and testing. We intend for this risk assessment of how various classes of AI might interact with various classes of disability to provide a roadmap for future research that is needed to gather data, test these hypotheses, and build more inclusive algorithms.

Sense and Accessibility

“…We also note that participants did not mention interactions with sophisticated AI or machine learning systems; it is unclear whether this is because such systems are not yet widely deployed and therefore were truly un-encountered by our respondents, or whether participants did yet not have a mental model of when they were interacting with AI-powered systems. For example, some technologists have reported challenges with self-driving car technologies [52] and robots [34] with respect to recognition of people with physical diferences, or other types of AI systems misrecognizing input from this demographic [29], but these issues were not surfaced in our study; re-visiting these topics over the next few years as more sophisticated ML systems become widely deployed in society is an important avenue for future work.…”

“…Although some sensing systems are powered by simple indicators and heuristics, many are powered by AI algorithms, and even simple sensor readings may feed into AI systems, particularly as sensing devices are increasingly networked (e.g., the IoT). Recently, researchers have issued calls-to-action to investigate how various AI technologies (such as vision, speech, text, and integrative AI systems) might require scrutiny regarding their impacts on people with disabilities, including issues such as whether such technologies are trained on inclusive datasets, whether they have reasonable accuracy for people with disabilities, and whether they might further societal biases against disadvantaged groups [6,19,29,32,53,61]. This paper adds to our understanding of issues at the intersection of AI and disability by providing evidence of the ways in which status quo sensing systems pose challenges for people with physical disabilities.…”

“…A 2017 report from the University of New Hampshire's Institute on Disability found that overall about 6.6% of Americans have an ambulatory disability, rising to 22.5% when considering only those aged 65 and over [39], and the U.S. Census Bureau issued a 2019 report about workers who fle for disability benefts, noting that 34.4% of disabled U.S. workers have ambulatory disabilities [56]. Researchers have recently issued calls-to-action to investigate whether AI systems properly recognize people with physical disabilities [29,61]; in this paper, we contribute to the growing body of knowledge in this area by investigating the types of challenges that sensing systems pose for the substantial subset of the population comprised of people with physical disabilities.…”

Machine Learning in Mental Health

“…While these metrics present established performance measures, as aggregate measures, they can hide varying model performance or biases across different population groups (cf. [ 73 ]). This also emphasizes the need to ensure that existing datasets capture the complexity of the real world (e.g .…”

IncluSet: A Data Surfacing Repository for Accessibility Datasets

“…For instance, creating datasets for emotion recognition for people who are Deaf 1 requires sign language fuency since, in addition to emotion, facial expressions are an inherent part of the language often used to convey syntax when signing [22]. And more importantly, there are privacy and ethical concerns for creating and sharing accessibility datasets 2 as people who have distinct data patterns may be more susceptible to data abuse and misuse [10,13,30].…”