Audio-visual speech understanding in simulated telephony applications by individuals with hearing loss

Kozma-Spytek, Linda; Tucker, Paula E.; Vogler, Christian

doi:10.1145/2513383.2517032

Cited by 6 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even though being able to see a talker’s face during video conferencing can be helpful to improve speech understanding (Gosselin & Gagné 2011; Devesse et al 2018), the degraded facial cues or auditory cues and even mismatch of audio-video information could still be misleading and difficult for this population. For example, Kozma-Spytek et al (2013) reported that, for people with hearing loss to understand spoken language in videotelephony, even small audio-video asynchrony can lead to great negative impact. Therefore, it warrants more investigation to study the text benefit in listeners with hearing loss when there are three sources of information in two modalities (visual text supplementation, visual facial cues, and auditory signal).…”

Section: Discussionmentioning

confidence: 99%

Benefits of Text Supplementation on Sentence Recognition and Subjective Ratings With and Without Facial Cues for Listeners With Normal Hearing

et al. 2022

View full text Add to dashboard Cite

Objectives: Recognizing speech through telecommunication can be challenging in unfavorable listening conditions. Text supplementation or provision of facial cues can facilitate speech recognition under some circumstances. However, our understanding of the combined benefit of text and facial cues in telecommunication is limited. The purpose of this study was to investigate the potential benefit of text supplementation for sentence recognition scores and subjective ratings of spoken speech with and without facial cues available. Design: Twenty adult females (M = 24 years, range 21 to 29 years) with normal hearing performed a sentence recognition task and also completed a subjective rating questionnaire in 24 conditions. The conditions varied by integrity of the available facial cues (clear facial cues, slight distortion facial cues, great distortion facial cues, no facial cues), signal-to-noise ratio (quiet, +1 dB, −3 dB), and text availability (with text, without text). When present, the text was an 86 to 88% accurate transcription of the auditory signal presented at a 500 ms delay relative to the auditory signal. Results: The benefits of text supplementation were largest when facial cues were not available and when the signal-to-noise ratio was unfavorable. Although no recognition score benefit was present in quiet, recognition benefit was significant in all levels of background noise for all levels of facial cue integrity. Moreover, participant subjective ratings of text benefit were robust and present even in the absence of recognition benefit. Consistent with previous literature, facial cues were beneficial for sentence recognition scores in the most unfavorable signal-to-noise ratio, even when greatly distorted. It is interesting that, although all levels of facial cues were beneficial for recognition scores, participants rated a significant benefit only with clear facial cues. Conclusions: The benefit of text for auditory-only and auditory-visual speech recognition is evident in recognition scores and subjective ratings; the benefit is larger and more robust for subjective ratings than for scores. Therefore, text supplementation might provide benefit that extends beyond speech recognition scores. Combined, these findings support the use of text supplementation in telecommunication, even when facial cues are concurrently present, such as during teleconferencing or watching television.

show abstract

Section: Discussionmentioning

confidence: 99%

Benefits of Text Supplementation on Sentence Recognition and Subjective Ratings With and Without Facial Cues for Listeners With Normal Hearing

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Often, these activities take place in settings referred to as hackerspaces, which enable communities and groups to have a physical space to bring people together in implementing ideas [41,72]. Individuals often take part in these activities for purposes other than financial gain [40] and can share these designs online and opensource [11]. DIY and hacking should not simply be perceived as a hobbyist or leisure practice, but as a professionalizing field functioning in parallel to research and industry labs [41].…”

Section: Diy and Hacking Assistive Technologiesmentioning

confidence: 99%

Social, Cultural and Systematic Frustrations Motivating the Formation of a DIY Hearing Loss Hacking Community

O’Kane

Aliomar

Zheng

et al. 2019

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Research on attitudes to assistive technology (AT) has shown both the positive and negative impact of these technologies on quality of life. Building on this research, we examine the sociocultural and technological frustrations with hearing loss (HL) technologies that motivate personal approaches to solving these issues. Drawing on meet-up observations and contextual interview data, we detail participants' experiences of and attitudes towards hearing AT that influences hacking hearing loss. Hearing AT is misunderstood as a solution to the impairment, influencing one-to-one interactions, cultural norms, and systematic frustrations. Participants' exasperation with the slow development of top-down solutions has led some members to design and develop their own personalised solutions. Beyond capturing a segment of the growing DIY health and wellbeing phenomenon, our findings extend beyond implications for design to present recommendations for the hearing loss industry, policy makers, and importantly, for researchers engaging with grassroots DIY health movements. CCS CONCEPTS • Human centered computing → Human computer interaction (HCI); Field Studies; Empirical studies in HCI; • Human centered computing → Collaborative and social computing; • Human centered computing → Accessibility; Empirical studies in accessibility.

show abstract

Voice Telephony for Individuals with Hearing Loss

Kozma-Spytek

Tucker

Vogler

2019

The 21st International ACM SIGACCESS Conference on Computers and Accessibility

Self Cite

View full text Add to dashboard Cite

This paper describes three studies conducted with a total of 114 individuals with hearing loss and 12 hearing controls, with the goal of investigating the impact of audio quality parameters on the accessibility of voice telecommunications. Three categories of parameters are covered: (1) narrowband (NB) versus wideband (WB) audio; (2) encoding audio at varying bit rates, ranging from typical rates used in today's telecom networks to the highest quality supported by these audio codecs; and (3) absence of packet loss to worst-case packet loss in VoIP telephony. With WB audio, individuals with hearing loss exhibit better speech recognition, expend less perceived mental effort, and rate speech quality higher than with NB audio. Bit rate affects speech recognition for NB audio, and speech quality ratings for both NB and WB audio. Packet loss affects all of speech recognition, mental effort, and speech quality ratings. WB versus NB audio also affects hearing individuals, especially under packet loss.

show abstract

Audio-visual speech understanding in simulated telephony applications by individuals with hearing loss

Cited by 6 publications

References 14 publications

Benefits of Text Supplementation on Sentence Recognition and Subjective Ratings With and Without Facial Cues for Listeners With Normal Hearing

Benefits of Text Supplementation on Sentence Recognition and Subjective Ratings With and Without Facial Cues for Listeners With Normal Hearing

Social, Cultural and Systematic Frustrations Motivating the Formation of a DIY Hearing Loss Hacking Community

Voice Telephony for Individuals with Hearing Loss

Contact Info

Product

Resources

About