Subjective Evaluation of a Noise-Reduced Training Target for Deep Neural Network-Based Speech Enhancement

Abstract

Speech enhancement systems aim to improve the quality and intelligibility of noisy speech. In this study, we compare two speech enhancement systems based on deep neural networks. The speech intelligibility and quality of both systems was evaluated subjectively, by a Speech Recognition Test based on Hagerman sentences and a translation of the ITU-T P.835 recommendation, respectively. Results were compared with the objective measures STOI and POLQA. Neither STOI nor POLQA reliably predicted subjective results. While STOI anticipated improvement, subjective results for both models showed degradation of speech intelligibility. POLQA results were overall hardly affected, while the subjective results showed significant changes in overall quality, both positive and negative, in many of the tests. One of the systems was trained to remove all noise; a strategy that is common in speech enhancement systems found in the literature. The other system was trained to only reduce the noise such that the signal-to-noise ratio increased with 10 dB. The latter system subjectively outperformed the system that attempted to remove noise completely. From this, we conclude that objective evaluation cannot replace subjective evaluation until a measure that reliably predicts intelligibility and quality for deep neural network based systems has been identified. Results further indicate that it may be beneficial to move away from more aggressive noise removal strategies towards noise reduction strategies that cause less speech distortion.

RETRACTED: Speech enhancement method using deep learning approach for hearing-impaired listeners

“…[13][14][15]53 In the current trend, machine learning approaches 16 have proved its great strength in speech intelligibility improvement for CI users, 17 NH listeners, HI listeners, [18][19][20] and medical signals. 6,[21][22][23] Normally, based on the incoming signal, gain function for noise and speech statistics is estimated. Nonetheless, the optimal gain function was estimated using the traditional machine learning approaches by means of incorporating prior knowledge of speech and noise patterns.…”

Improved Wiener Filter Algorithm for Speech Enhancement

“…This paper uses the amplitude spectrum under two hypothetical conditions to carry out simulation experiments, and compares the results with the ideal state, we find the speech intelligibility has indeed improved through the objective evaluation index of speech intelligibility, and the improvement of the methods has been successfully applied to reality. In recent years, deep neural network algorithms have also been applied to the field of speech enhancement [16]. Speech enhancement algorithms based on improved speech intelligibility also have good development prospects [17,18].…”

Real-Time Speech Enhancement Algorithm Based on Attention LSTM

“…For these three indicators, the higher the value is, the higher the speech quality. Some scholars think that objective evaluation cannot replace subjective evaluation [43]. So, in addition to the objective measurements, subjective listening tests are performed to evaluate the perceptual quality of enhanced speech signals.…”