A new technology to compensate degeneration of hearing intelligibility for elderly individuals: Development of a portable real-time speech rate conversion system

Abstract

This paper presents a portable real-time speech rate conversion system for compensating degeneration of hearing intelligibility of elderly listeners who are suffering from listening to rapid speech. This system enables an elderly user to convert a speech rate as desired by him/herself on real time, with invariance in pitch as well as small impairments in quality. Conventional hearing aids are focusing on compensation of decreased hearing abilities of peripheral auditory pathways in the frequency area, while the new system tries to compensate for them with central auditory pathways in the temporal area. When this system is applied to speech accompanied with a picture, temporal discrepancy between the converted voice and the picture can be absorbed by changing the speech rate at every pitch period along a monotone decreasing function from slow to fast. The system is 180×130×65 mm in size. Another feature allows a user to change the speech rate quickly whenever he/she wants to convert at any stage of conversion. This system can be applied to both the Japanese spoken language and foreign spoken languages, and can be used as a listening aid to learn foreign languages.

Effect of speed difference between time-expanded speech and moving image of talker’s face on word intelligibility

“…In these studies, the amount of asynchrony between auditory and visual speech signals was constant from the onset to the offset of the signals. However, in the multimodal speech-rate conversion system [3,4], the audio lag grows progressively toward the end, whereas the auditory and visual signals are synchronized at the beginning because only auditory signal was time-expanded. In this situation, a visual advantage may no longer occur in spoken language comprehension.…”

“…However, there exists an asynchrony between auditory and visual speech signals in this system [3,4]. Earlier studies have investigated a temporal window of auditory-visual speech integration during which auditory and visual speech signals are integrated, and have suggested that the size of the temporal window is around 200 ms when an auditory signal lags a visual signal [5][6][7][8][9][10][11][12][13].…”

“…The cue of moving image of talker's face is usually called "lip-reading" information, and this cue improves speech intelligibility for normal hearing listeners under a low signal-to-noise ratio (S/N) condition [2]. Based on these knowledge, a speech rate conversion technique has been proposed and applied to broadcasting systems [3,4].…”

“…In this system [3,4], speech rate conversion is realized by slowing down "only" the speech signal without accumulating a time delay. In particular, the total duration is kept constant by deleting the pauses between phrases to an amount equal to the duration of the speech expanded.…”

Development of Digital Hearing Aids

“…The possibility of addressing this problem with technology has generally seemed far fetched because if a system did slow down the delivery of speech acoustics it would seem to potentially "de-synchronize" the visual cues associated with speech production. Nevertheless such a system has been recently developed for addressing the excessive speech rate complaints of elderly listeners (Miyasaka et al, 1996). The developers say the unit, which allows the user to quickly make adjustments to the processing rate, is useful for learning a foreign language, as well as an auditory rehabilitation tool.…”