A deep learning based segregation algorithm to increase speech intelligibility for hearing-impaired listeners in reverberant-noisy conditions

Abstract

Recently, deep learning based speech segregation has been shown to improve human speech intelligibility in noisy environments. However, one important factor not yet considered is room reverberation, which characterizes typical daily environments. The combination of reverberation and background noise can severely degrade speech intelligibility for hearing-impaired (HI) listeners. In the current study, a deep learning based time-frequency masking algorithm was proposed to address both room reverberation and background noise. Specifically, a deep neural network was trained to estimate the ideal ratio mask, where anechoic-clean speech was considered as the desired signal. Intelligibility testing was conducted under reverberant-noisy conditions with reverberation time T 60 ¼ 0.6 s, plus speech-shaped noise or babble noise at various signal-to-noise ratios. The experiments demonstrated that substantial speech intelligibility improvements were obtained for HI listeners. The algorithm was also somewhat beneficial for normal-hearing (NH) listeners. In addition, sentence intelligibility scores for HI listeners with algorithm processing approached or matched those of young-adult NH listeners without processing. The current study represents a step toward deploying deep learning algorithms to help the speech understanding of HI listeners in everyday conditions. V

A deep learning algorithm to increase intelligibility for hearing-impaired listeners in the presence of a competing talker and reverberation

“…Fortunately, deep learning algorithms based on timefrequency (T-F) masking have been successful at improving intelligibility under these varied conditions, and particularly for the population of greatest need-those with sensorineural hearing loss who wear hearing aids. Intelligibility improvements have been observed for backgrounds consisting of steady-state noise (Healy et al, 2013;Healy et al, 2014;Monaghan et al, 2017;Zhao et al, 2018), speech babble (Healy et al, 2013;Healy et al, 2014;Healy et al, 2015;Chen et al, 2016;Monaghan et al, 2017;Bentsen et al, 2018;Zhao et al, 2018), and cafeteria-noise recordings (Healy et al, 2015;Chen et al, 2016;Zhao et al, 2018).…”

“…But the two types of distortion disrupt the acoustic speech signal in different ways, leading to different effects on speech perception (producing for example different patterns of vowel-perception errors in human listeners, N ab elek and Dagenais, 1986). Likely due to the considerable challenge associated with addressing these concurrent distortions, only recently reported is the first demonstration of intelligibility improvement resulting from a single-microphone (monaural) algorithm in reverberant-noisy conditions (Zhao et al, 2018). In this report, deep learning was used to estimate T-F masks for sentences corrupted by reverberation plus speech-shaped noise or reverberation plus multi-talker babble.…”

“…Said differently, what signal should the algorithm be trained to extract? In Zhao et al (2018), the decision was made to remove both reverberation and noise, and aim for reverberation-free, noise-free speech. This method, which can be referred to as the direct-sound (DS) target, is one of two processing strategies employed in the current study.…”

“…First, when background interference is absent, human listeners can tolerate a substantial amount of reverberation. This is particularly true for NH listeners, but also to some extent for those with hearing loss or cochlear implants (see the brief discussion by Zhao et al, 2018). Second, the computational challenge associated with accurately estimating the R target is likely reduced relative to that associated with accurately estimating the DS target, because de-reverberation of the speech of interest is not performed in the former and the output sound more closely resembles the original input sound.…”

A Study of Joint Effect on Denoising Techniques and Visual Cues to Improve Speech Intelligibility in Cochlear Implant Simulation

“…The SE process consists of two parts: to enhance the intelligibility and quality of processed speech, and to reduce the noises in the background. Previous well-established algorithms have helped improve the SE in CI users [37], [38], [29], [39], [40], [41], [42], [43] but there are only few studies with a newly upgrading deep-learning-based algorithm. Traditional SE methods are based on identifying the difference between clean and noisy speech [44], [45], [46], [47], [48], [49].…”