A near-end listening enhancement system by RNN-based noise cancellation and speech modification

Li, Gang; Hu, Ruimin; Wang, Xiaochen; Zhang, Rui

doi:10.1007/s11042-018-6947-8

Cited by 8 publications

(5 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These complex and variable noise environments bring great interference to communication. As shown in Figure 1, the interference of environmental noise in mobile communication mainly comes from two stages: the "talking stage" (in the far-end) and the "listening stage" (in the nearend) [1]. The noise in the "speaking phase" has been relatively well suppressed by the development of hardware and software for a long time, which is represented by speech enhancement (SE) [2,3].…”

Section: Introductionmentioning

confidence: 99%

D2StarGAN: A Near-Far End Noise Adaptive StarGAN for Speech Intelligibility Enhancement

Li,

Zhu,

Zhao

2023

Electronics

View full text Add to dashboard Cite

When using mobile communication, the voice output from the device is already relatively clear, but in a noisy environment, it is difficult for the listener to obtain the information expressed by the speaker with clarity. Consequently, speech intelligibility enhancement technology has emerged to help alleviate this problem. Speech intelligibility enhancement (IENH) is a technique that enhances speech intelligibility during the reception phase. Previous research has focused on IENH through normal versus different levels of Lombardic speech conversion, inspired by a well-known acoustic mechanism called the Lombard effect. However, these methods often lead to speech distortion and impair the overall speech quality. To address the speech quality degradation problem, we propose an improved (StarGAN)-based IENH framework by combining StarGAN networks with the dual discriminator idea to construct the conversion framework. This approach offers two main advantages: (1) Addition of a speech metric discriminator on top of StarGAN to optimize multiple intelligibility and quality-related metrics simultaneously; (2) a framework that is adaptive to different distal and proximal noise levels with different noise types. Experimental results from objective experiments and subjective preference tests show that our approach outperforms the baseline approach, and these enable IENH to be more widely used.

show abstract

Section: Introductionmentioning

confidence: 99%

D2StarGAN: A Near-Far End Noise Adaptive StarGAN for Speech Intelligibility Enhancement

Li,

Zhu,

Zhao

2023

Electronics

View full text Add to dashboard Cite

show abstract

“…We note for near-end listening with headphones, adaptive noise cancellation (ANC) techniques [45], [46] may be employed, which -instead of processing the target speech signal -aim at adaptively cancelling the noise by adding an antiphase noise component to the speech signal before playout in the noisy environment [47], [48]. However, ANC with classic adaptive filtering has generally been insufficient for improving intelligibility outside headphone use until the use of DNNs [47]. Hence, NLE based on speech modification is still the predominant approach.…”

Section: Introductionmentioning

confidence: 99%

“…Common for the objective SI metrics such as SII and the Glimpse model [29] is that audibility is the decisive factor of intelligibility. Therefore, the principle for all NLE algorithms is to adjust the SNR for the perceptually important parts of the speech [47]. Hence, the simplest solution to the NLE problem is to increase the power of the clean speech signal until the noise is sufficiently masked, i.e., increasing the SNR (almost indefinitely).…”

Section: Introductionmentioning

confidence: 99%

Minimum Processing Near-end Listening Enhancement

Fuglsig¹,

Jensen²,

Bertelsen³

et al. 2022

Preprint

View full text Add to dashboard Cite

The intelligibility and quality of speech from a mobile phone or public announcement system are often affected by background noise in the listening environment. By preprocessing the speech signal it is possible to improve the speech intelligibility and quality -this is known as near-end listening enhancement (NLE). Although, existing NLE techniques are able to greatly increase intelligibility in harsh noise environments, in favorable noise conditions the intelligibility of speech reaches a ceiling where it cannot be further enhanced. Actually, the focus of existing methods solely on improving the intelligibility causes unnecessary processing of the speech signal and leads to speech distortions and quality degradations. In this paper, we provide a new rationale for NLE, where the target speech is minimally processed in terms of a processing penalty, provided that a certain performance constraint, e.g., intelligibility, is satisfied. We present a closed-form solution for the case where the performance criterion is an intelligibility estimator based on the approximated speech intelligibility index and the processing penalty is the mean-square error between the processed and the clean speech. This produces an NLE method that adapts to changing noise conditions via a simple gain rule by limiting the processing to the minimum necessary to achieve a desired intelligibility, while at the same time focusing on quality in favorable noise situations by minimizing the amount of speech distortions. Through simulation studies, we show the proposed method attains speech quality on par or better than existing methods in both objective measurements and subjective listening tests, whilst still sustaining objective speech intelligibility performance on par with existing methods.

show abstract

“…We note for near-end listening with headphones, adaptive noise cancellation (ANC) techniques [37], [38] may be employed, which -instead of processing the target speech signal -aim at adaptively cancelling the noise by adding an anti-phase noise component to the speech signal before playout in the noisy environment [39], [40]. However, ANC with classic adaptive filtering has generally been insufficient for improving intelligibility outside headphone use until the use of DNNs [39]. Hence, NLE based on speech modification is still the predominant approach.…”

mentioning

confidence: 99%

“…Common for the objective SI metrics such as SII and the Glimpse model [41] is that audibility is the decisive factor of intelligibility. Therefore, the principle for all NLE algorithms is to adjust the SNR for the perceptually important parts of the speech [39]. Hence, the simplest solution to the NLE problem is to increase the power of the clean speech signal until the noise is sufficiently masked, i.e., increasing the SNR (almost indefinitely).…”

mentioning

confidence: 99%

Minimum Processing Near-End Listening Enhancement

Fuglsig

Jensen

Bertelsen³

et al. 2023

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

The intelligibility and quality of speech from a mobile phone or public announcement system are often affected by background noise in the listening environment. By pre-processing the speech signal it is possible to improve the speech intelligibility and quality -this is known as near-end listening enhancement (NLE). Although, existing NLE techniques are able to greatly increase intelligibility in harsh noise environments, in favorable noise conditions the intelligibility of speech reaches a ceiling where it cannot be further enhanced. Actually, the focus of existing methods solely on improving the intelligibility causes unnecessary processing of the speech signal and leads to speech distortions and quality degradations. In this article, we provide a new rationale for NLE, where the target speech is minimally processed in terms of a processing penalty, provided that a certain performance constraint, e.g., intelligibility, is satisfied. We present a closed-form solution for the case where the performance criterion is an intelligibility estimator based on the approximated speech intelligibility index and the processing penalty is the mean-square error between the processed and the clean speech. This produces an NLE method that adapts to changing noise conditions via a simple gain rule by limiting the processing to the minimum necessary to achieve a desired intelligibility, while at the same time focusing on quality in favorable noise situations by minimizing the amount of speech distortions. Through simulation studies, we show the proposed method attains speech quality on par or better than existing methods in both objective measurements and subjective listening tests, whilst still sustaining objective speech intelligibility performance on par with existing methods.

show abstract

A near-end listening enhancement system by RNN-based noise cancellation and speech modification

Cited by 8 publications

References 33 publications

D2StarGAN: A Near-Far End Noise Adaptive StarGAN for Speech Intelligibility Enhancement

D2StarGAN: A Near-Far End Noise Adaptive StarGAN for Speech Intelligibility Enhancement

Minimum Processing Near-end Listening Enhancement

Minimum Processing Near-End Listening Enhancement

Contact Info

Product

Resources

About