Predicting speech intelligibility with deep neural networks

Spille, Constantin; Ewert, Stephan D.; Kollmeier, Birger; Meyer, Bernd T.

doi:10.1016/j.csl.2017.10.004

Cited by 72 publications

(56 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, a number of works have attempted to partly or fully predict intelligibility using data-driven methods. One approach in this direction has been to use an Automatic Speech Recognition (ASR) system to transcribe degraded sentences, using the error rate as a measure of intelligibility [48], [49]. Another data-driven approach has been to non-intrusively estimate the output of an intrusive SIP algorithm [50], [51], [52], [53].…”

Section: Introductionmentioning

confidence: 99%

Nonintrusive Speech Intelligibility Prediction Using Convolutional Neural Networks

Andersen

Haan

Jensen

2018

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Speech Intelligibility Prediction (SIP) algorithms are becoming popular tools within the development and operation of speech processing devices and algorithms. However, many SIP algorithms require knowledge of the underlying clean speech; a signal that is often not available in real-world applications. This has led to increased interest in non-intrusive SIP algorithms, which do not require clean speech to make predictions. In this paper we investigate the use of Convolutional Neural Networks (CNNs) for non-intrusive SIP. To do so, we utilize a CNN architecture that shows similarities to existing SIP algorithms, in terms of computational structure, and which allows for easy and meaningful visualization and interpretation of trained weights. We evaluate this architecture using a large dataset obtained by combining datasets from the literature. The proposed method shows high prediction performance when compared with four existing intrusive and non-intrusive SIP algorithms. This demonstrates the potential of deep learning for speech intelligibility prediction.

show abstract

Section: Introductionmentioning

confidence: 99%

Nonintrusive Speech Intelligibility Prediction Using Convolutional Neural Networks

Andersen

Haan

Jensen

2018

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…Despite the breakthroughs of neural networks in so many areas, to date, only a handful of neural network-based models have been proposed [9,10,11]. To the best of our knowledge, even the most recent methods to predict MOS present serious limitations.…”

Section: Introductionmentioning

confidence: 99%

“…To the best of our knowledge, even the most recent methods to predict MOS present serious limitations. First, most of them are developed to measure intelligibility [10], which is just one aspect of audio quality [3]. Second, these neural network-based models are trained on a limited number of conditions, usually with no interaction between different impairments, which is quite unrealistic and rarely happens in everyday scenarios.…”

Section: Introductionmentioning

confidence: 99%

Non-intrusive Speech Quality Assessment Using Neural Networks

Avila

Gamper

Reddy

et al. 2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Estimating the perceived quality of an audio signal is critical for many multimedia and audio processing systems. Providers strive to offer optimal and reliable services in order to increase the user quality of experience (QoE). In this work, we present an investigation of the applicability of neural networks for non-intrusive audio quality assessment. We propose three neural network-based approaches for mean opinion score (MOS) estimation. We compare our results to three instrumental measures: the perceptual evaluation of speech quality (PESQ), the ITU-T Recommendation P.563, and the speech-to-reverberation energy ratio. Our evaluation uses a speech dataset contaminated with convolutive and additive noise, labeled using a crowd-based QoE evaluation, evaluated with Pearson correlation with MOS labels, and mean-squared-error of the estimated MOS. Our proposed approaches outperform the aforementioned instrumental measures, with a fully connected deep neural network using Mel-frequency features providing the best correlation (0.87) and the lowest mean squared error (0.15).Index Terms-Audio quality assessment, speech quality assessment, deep neural network * Work on this project performed as an intern at Microsoft Research Labs, Redmond, WA.

show abstract

“…In [3], the speech intelligibility is predicted using convolutional neural network which is trained with measured intelligibility scores that humans listen and evaluate. The work in [4] presented the method of speech intelligibility prediction by using automatic speech recognition (ASR) system based deep neural networks. Recently, the non-intrusive speech intelligibility estimation method based on a recurrent neural network (RNN) with a mel-frequency cepstrum coefficient (MFCC) vector was proposed [5].…”

Section: Introductionmentioning

confidence: 99%

A Non-Intrusive Speech Intelligibility Estimation Method Based on Deep Learning Using Autoencoder Features

Kim

Yun

Lee

et al. 2020

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

This paper presents a deep learning-based non-intrusive speech intelligibility estimation method using bottleneck features of autoencoder. The conventional standard non-intrusive speech intelligibility estimation method, P.563, lacks intelligibility estimation performance in various noise environments. We propose a more accurate speech intelligibility estimation method based on long-short term memory (LSTM) neural network whose input and output are an autoencoder bottleneck features and a short-time objective intelligence (STOI) score, respectively, where STOI is a standard tool for measuring intrusive speech intelligibility with reference speech signals. We showed that the proposed method has a superior performance by comparing with the conventional standard P.563 and mel-frequency cepstral coefficient (MFCC) feature-based intelligibility estimation methods for speech signals in various noise environments.

show abstract

Predicting speech intelligibility with deep neural networks

Cited by 72 publications

References 33 publications

Nonintrusive Speech Intelligibility Prediction Using Convolutional Neural Networks

Nonintrusive Speech Intelligibility Prediction Using Convolutional Neural Networks

Non-intrusive Speech Quality Assessment Using Neural Networks

A Non-Intrusive Speech Intelligibility Estimation Method Based on Deep Learning Using Autoencoder Features

Contact Info

Product

Resources

About