Evaluating Smartwatch-based Sound Feedback for Deaf and Hard-of-hearing Users Across Contexts

Goodman, Steven; Kirchner, Susanne; Guttman, Rose; Jain, Dhruv; Froehlich, Jon E.; Findlater, Leah

doi:10.1145/3313831.3376406

Cited by 16 publications

(10 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To the best of our knowledge, recent studies with high-end sensory devices typically have sample sizes with less than 50 participants (N = 19, M = 20.15, SD = 10.61) due to natural budget constraints. In addition to their usage in the ESM context, these devices are emerging as convenient and applicable tools for data collection and intervention in other domains such as in health (see [47][48][49][50]) and in cyber security (see [44,51]) as well.…”

Section: Related Workmentioning

confidence: 99%

Assessing the Influence of Physical Activity Upon the Experience Sampling Response Rate on Wrist-Worn Devices

Khanshan

Gorp

Nuijten

et al. 2021

IJERPH

View full text Add to dashboard Cite

The Experience Sampling Method (ESM) is gaining ground for collecting self-reported data from human participants during daily routines. An important methodological challenge is to sustain sufficient response rates, especially when studies last longer than a few days. An obvious strategy is to deliver the experiential questions on a device that study participants can access easily at different times and contexts (e.g., a smartwatch). However, responses may still be hampered if the prompts are delivered at an inconvenient moment. Advances in context sensing create new opportunities for improving the timing of ESM prompts. Specifically, we explore how physiological sensing on commodity-level smartwatches can be utilized in triggering ESM prompts. We have created Experiencer, a novel ESM smartwatch platform that allows studying different prompting strategies. We ran a controlled experiment (N=71) on Experiencer to study the strengths and weaknesses of two sampling regimes. One group (N=34) received incoming notifications while resting (e.g., sedentary), and another group (N=37) received similar notifications while being active (e.g., running). We hypothesized that response rates would be higher when experiential questions are delivered during lower levels of physical activity. Contrary to our hypothesis, the response rates were found significantly higher in the active group, which demonstrates the relevance of studying dynamic forms of experience sampling that leverage better context-sensitive sampling regimes. Future research will seek to identify more refined strategies for context-sensitive ESM using smartwatches and further develop mechanisms that will enable researchers to easily adapt their prompting strategy to different contextual factors.

show abstract

Section: Related Workmentioning

confidence: 99%

Assessing the Influence of Physical Activity Upon the Experience Sampling Response Rate on Wrist-Worn Devices

Khanshan

Gorp

Nuijten

et al. 2021

IJERPH

View full text Add to dashboard Cite

show abstract

“…While accounting for the diverse perspectives of DHH people, prior work also highlights several general preferences among DHH users. Overall, the most desired sound characteristic is identity, which users prioritize when compared to other characteristics like volume or duration [8,22,27,54]. When discussing sounds of interest, DHH users generally rank awareness of urgent sounds (e.g., safety-related alarms, sirens) as most important, followed by sounds that indicate others' presence (e.g., door knocks, footsteps) and appliance alerts (e.g., oven timers, pop-up toasters) [8,22,54,67].…”

Section: Diversity Of Sound Awareness Needsmentioning

confidence: 99%

“…When discussing sounds of interest, DHH users generally rank awareness of urgent sounds (e.g., safety-related alarms, sirens) as most important, followed by sounds that indicate others' presence (e.g., door knocks, footsteps) and appliance alerts (e.g., oven timers, pop-up toasters) [8,22,54,67]. Additionally, the relevance of sound information may change as the user moves between social contexts (e.g., family vs. strangers) [8,22,36] and physical locations (e.g., at home vs. while mobile) [27,54]. For example, in the home, sound identity may be adequate [37], while directional indicators are important when mobile [27].…”

Section: Diversity Of Sound Awareness Needsmentioning

confidence: 99%

“…Additionally, the relevance of sound information may change as the user moves between social contexts (e.g., family vs. strangers) [8,22,36] and physical locations (e.g., at home vs. while mobile) [27,54]. For example, in the home, sound identity may be adequate [37], while directional indicators are important when mobile [27]. DHH users have frequently expressed interest in sounds that are specifc to their life (e.g., babies and children [54], name calls [8]) and fne-tuned sounds in their home [37], which points to the need for end users to be able to personalize their sound awareness tools-the focus of our study.…”

Section: Diversity Of Sound Awareness Needsmentioning

confidence: 99%

“…Personalization is essential to provide context-specifc support and meet the wide-ranging needs of the DHH community [22,27,53,55,58]. Some projects have explored options for DHH users to flter notifcations for certain sounds [27,37,38], but they stopped short of adding or modifying sound classes through user-provided recordings. Bragg et al [8] designed a personalizable sound recognition smartphone app and recruited 12 DHH participants for a Wizard-of-Oz usability study.…”

Section: Sound Awareness Toolsmentioning

confidence: 99%

See 2 more Smart Citations

Toward User-Driven Sound Recognizer Personalization with People Who Are d/Deaf or Hard of Hearing

Goodman

Liu

Jain

et al. 2021

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

Self Cite

View full text Add to dashboard Cite

Automated sound recognition tools can be a useful complement to d/Deaf and hard of hearing (DHH) people's typical communication and environmental awareness strategies. Pre-trained sound recognition models, however, may not meet the diverse needs of individual DHH users. While approaches from human-centered machine learning can enable non-expert users to build their own automated systems, end-user ML solutions that augment human sensory abilities present a unique challenge for users who have sensory disabilities: how can a DHH user, who has difficulty hearing a sound themselves, effectively record samples to train an ML system to recognize that sound? To better understand how DHH users can drive personalization of their own assistive sound recognition tools, we conducted a three-part study with 14 DHH participants: (1) an initial interview and demo of a personalizable sound recognizer, (2) a week-long field study of in situ recording, and (3) a follow-up interview and ideation session. Our results highlight a positive subjective experience when recording and interpreting training data in situ, but we uncover several key pitfalls unique to DHH users---such as inhibited judgement of representative samples due to limited audiological experience. We share implications of these results for the design of recording interfaces and human-the-the-loop systems that can support DHH users to build sound recognizers for their personal needs.

show abstract

Inclusive Augmented and Virtual Reality: A Research Agenda

Creed,

Al-Kalbani,

Theil

et al. 2023

International Journal of Human–Computer Interaction

View full text Add to dashboard Cite

Evaluating Smartwatch-based Sound Feedback for Deaf and Hard-of-hearing Users Across Contexts

Cited by 16 publications

References 42 publications

Assessing the Influence of Physical Activity Upon the Experience Sampling Response Rate on Wrist-Worn Devices

Assessing the Influence of Physical Activity Upon the Experience Sampling Response Rate on Wrist-Worn Devices

Toward User-Driven Sound Recognizer Personalization with People Who Are d/Deaf or Hard of Hearing

Inclusive Augmented and Virtual Reality: A Research Agenda

Contact Info

Product

Resources

About