Revisiting Blind Photography in the Context of Teachable Object Recognizers

Lee, Kyungjun; Hong, Jonggi; Pimento, Simone; Jarjue, Ebrima; Kacorri, Hernisa

doi:10.1145/3308561.3353799

Cited by 26 publications

(14 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fiebrink, for example, has shown considerable success in building digital music instruments that can be trained and respond to real-time embodied interaction by users [18]. Indeed, we find possible starting points in object recognizers which allow vision impaired users to build their own training sets [41], [49]. How such approaches might be incorporated into the mundane attunements between human actors, and those with different sensory capacities, presents an open question but one that seems in line with the premise of this humancentered machine learning and at least technically feasible.…”

Section: Salience In the Momentmentioning

confidence: 94%

“…The design and use of ATs is a now established thread of research in HCI. Relevant, for example, are projects that have used computer vision to support people with vision impairments to complete tasks like identifying objects, people, and the contents of photos on social media [8], [40], [41], [49], [53], [78], [88], [90], [91], [92], [93], [94], [95]. For example, VizWiz [8] allowed blind people to photograph images for algorithms or crowd workers to describe.…”

Section: Ai Ats and Social Interactionsmentioning

confidence: 99%

“…In the data driven approaches popular in contemporary AI systems and being introduced into ATs [4], [8], [27], [40], [41], [49], [53], [58], [78], [88], [90], [91], [92], [93], [94], [95] these mundane attunements present a challenge. When what matters in the work of access is open, attuned in response to so many contingencies, how do we find datasets that might afford aggregation in any meaningful sense and that lead to the identification of actionable patterns of interaction that aren't too general?…”

Section: Mundane Attunementmentioning

confidence: 99%

See 2 more Smart Citations

The Care Work of Access

Bennett

Rosner

Taylor

2020

Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Current approaches to AI and Assistive Technology (AT) often foreground task completion over other encounters such as expressions of care. Our paper challenges and complements such task-completion approaches by attending to the care work of access-the continual affective and emotional adjustments that people make by noticing and attending to one another. We explore how this work impacts encounters among people with and without vision impairments who complete tasks together. We find that bound up in attempts to get things done are concerns for one another and how well people are doing together. Reading this work through emerging disability studies and feminist STS scholarship, we account for two important forms of work that give rise to access: (1) mundane attunements and (2) noninnocent authorizations. Together these processes work as sensitizing concepts to help HCI scholars account for the ways that intelligent ATs both produce access while sometimes subverting people with disabilities.

show abstract

Section: Salience In the Momentmentioning

confidence: 94%

Section: Ai Ats and Social Interactionsmentioning

confidence: 99%

Section: Mundane Attunementmentioning

confidence: 99%

See 1 more Smart Citation

The Care Work of Access

Bennett

Rosner

Taylor

2020

Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…However, training an ML-enabled application as a personal assistive technology can itself be inaccessible when it requires skills and abilities similar to those the application is intended to support [23,40]. For example, a blind or visually impaired user is likely unable to use visual feedback when capturing images for personalizing an object recognizer-a challenge that Kacorri et al and others (e.g., [41,69]) frst examined via studies of users' needs in this context, and more recently began addressing through active feedback techniques to assist in image capture [49]. Indeed, in Nakao et al's [58] study of DHH users' technical understanding of ML, workshop participants struggled to choose acceptable sound samples for training due to a lack of non-auditory feedback.…”

Section: Human-centered Machine Learningmentioning

confidence: 99%

Toward User-Driven Sound Recognizer Personalization with People Who Are d/Deaf or Hard of Hearing

Goodman

Liu

Jain

et al. 2021

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

Automated sound recognition tools can be a useful complement to d/Deaf and hard of hearing (DHH) people's typical communication and environmental awareness strategies. Pre-trained sound recognition models, however, may not meet the diverse needs of individual DHH users. While approaches from human-centered machine learning can enable non-expert users to build their own automated systems, end-user ML solutions that augment human sensory abilities present a unique challenge for users who have sensory disabilities: how can a DHH user, who has difficulty hearing a sound themselves, effectively record samples to train an ML system to recognize that sound? To better understand how DHH users can drive personalization of their own assistive sound recognition tools, we conducted a three-part study with 14 DHH participants: (1) an initial interview and demo of a personalizable sound recognizer, (2) a week-long field study of in situ recording, and (3) a follow-up interview and ideation session. Our results highlight a positive subjective experience when recording and interpreting training data in situ, but we uncover several key pitfalls unique to DHH users---such as inhibited judgement of representative samples due to limited audiological experience. We share implications of these results for the design of recording interfaces and human-the-the-loop systems that can support DHH users to build sound recognizers for their personal needs.

show abstract

“…Technologies associated with AR have recently been used in a variety of systems intended to improve access to the world using smartphones. A number of systems have been developed to help blind people take better photos, generally by using automated approaches to assist in aiming the camera [29,36,65]. VizLens uses a combination of computer vision and crowdsourcing to recognize and guide a blind user through using an inaccessible physical interface [22,23].…”

Section: Ar For Making the World More Accessiblementioning

confidence: 99%

Making Mobile Augmented Reality Applications Accessible

Herskovitz

White

et al. 2020

Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility

View full text Add to dashboard Cite

Augmented Reality (AR) technology creates new immersive experiences in entertainment, games, education, retail, and social media. AR content is often primarily visual and it is challenging to enable access to it non-visually due to the mix of virtual and real-world content. In this paper, we identify common constituent tasks in AR by analyzing existing mobile AR applications for iOS, and characterize the design space of tasks that require accessible alternatives. For each of the major task categories, we create prototype accessible alternatives that we evaluate in a study with 10 blind participants to explore their perceptions of accessible AR. Our study demonstrates that these prototypes make AR possible to use for blind users and reveals a number of insights to move forward. We believe our work sets forth not only exemplars for developers to create accessible AR applications, but also a roadmap for future research to make AR comprehensively accessible. CCS CONCEPTS • Human-centered computing → Human computer interaction (HCI); Accessibility technologies; Mixed / augmented reality.

show abstract

Revisiting Blind Photography in the Context of Teachable Object Recognizers

Cited by 26 publications

References 38 publications

The Care Work of Access

The Care Work of Access

Toward User-Driven Sound Recognizer Personalization with People Who Are d/Deaf or Hard of Hearing

Making Mobile Augmented Reality Applications Accessible

Contact Info

Product

Resources

About