Evaluating non-speech sound visualizations for the deaf

Abstract

Sounds such as co-workers chatting nearby or a dripping faucet help us maintain awareness of and respond to our surroundings. Without a tool that communicates ambient sounds in a nonauditory manner, maintaining this awareness is difficult for people who are deaf. We present an iterative investigation of peripheral, visual displays of ambient sounds. Our major contributions are: (1) a rich understanding of what ambient sounds are useful to people who are deaf, (2) a set of visual and functional requirements for a peripheral sound display, based on feedback from people who are deaf, (3) lab-based evaluations investigating the characteristics of four prototypes, and (4) a set of design guidelines for successful ambient audio displays, based on a comparison of four implemented prototypes and user feedback. Our work provides valuable information about the sound awareness needs of the deaf and can help to inform further design of such applications.

Exiting the Cleanroom: On Ecological Validity and Ubiquitous Computing

“…Researchers have found sketches effective in needfinding studies to facilitate concrete comparisons between different designs and to help participants express their expectations for a display. Matthews, Fong, Ho-Ching, and Mankoff (2006c) conducted needfinding interviews and sketch studies that led to the IC2Hear sound awareness display. In this study, the sketches gave users semiconcrete display ideas to discuss.…”

Head-Mounted Display Visualizations to Support Sound Awareness for the Deaf and Hard of Hearing

“…Prior work on visual aids for persons with hearing loss has focused largely on non-speech sounds (e.g., an alarm or doorbell) presented on external displays such as desktops or mobile devices [11,17,18,27]. Though promising, these external displays require the user to turn their attention away from conversational partners.…”

“…For each dimension, we generated 3-12 specific visualizations, which were informed by known sound awareness needs [11,17,18] and our own experiences as persons with hearing loss. 1 We then performed two evaluations: a design probe study (Study 1) with 24 deaf and hard of hearing participants and a small, exploratory study (Study 2) of a proof-of-concept HMD prototype with four new participants.…”

VizWiz

“…VizWiz often provides answers faster, although the information necessary to answer a VizWiz question is embedded in the photo, whereas ChaCha and KGB are often used to ask questions that might require a web search. Other common remote services include relay services for deaf and hard of hearing people (which requires trained employees) [27], and the retroactive nearly real-time audio captioning by dedicated workers in Scribe4Me [23]. A user study of Scribe4Me found that participants felt waiting the required 3-5 minutes was too long because it "leaves one as an observer rather than an active participant."…”