Probabilistic non-intrusive quality assessment of speech for bounded-scale preference scores

Petkov, Petko N.; Kleijn, W. Bastiaan

doi:10.1109/qomex.2010.5516236

“…The work is based on traditional PLP model, bark spectrum and MFCC. Additionally, a probabilistic, non-intrusive method for quality assessment of speech that takes into consideration the bounded character of the preference scores was proposed in [23].The special achievement of this approach is that by using the Maximum likelihood estimation to learn the model parameters, the learning stage is efficiently even from a small amount of training data.…”

Section: Recent Findingsmentioning

confidence: 99%

On objective assessment of audio quality — A review

Zheng

¹

,

Zhu

²

,

Ye

³

2012

2012 International Conference on Audio, Language and Image Processing

2

0

View full text Add to dashboard Cite

Measurement of audio quality is of fundamental importance for numerous speech and audio processing applications, where the goal of quality assessment algorithms is to evaluate the quality in agreement with human's subjective feelings of audio quality. This paper includes a brief summary of the development of intrusive and non-intrusive audio quality assessment algorithms and outlines recent advancements, especially some new topics to be investigated presently. Moreover, according to the existing problems in this field, discussion of the future trend and challenges is also given. 1．IntroductionObjective assessment of audio quality brings a lot of convenience when ranking different sound processing equipments before launching them on the market as well as observing audio signals quality in on-line monitoring. The traditional way of audio quality assessment is to collect peoples' opinions on the quality in a wellcontrolled listening test, for example Recommendation ITU-R BS.1116 [1], which has limited applications for too expensive and time-consuming. Therefore, objective audio quality measurement is naturally desired. Early methods, like Signal-to-Noise-Ratio (S/N) or TotalHarmonic-Distortion (THD) are unable to emulate human's judgments especially when the methods are applied to non-linear and non-stationary modern codec [2]. As for these disadvantages, lots of researchers studied perceptual quality assessment and these works have led to several ITU standards. Among these standards, the International Telecommunications Union (ITU) standard for audio quality (BS.1387)[3], also called perceptual evaluation of audio quality (PEAQ) has been the only available standardized method for the purpose of audio quality assessment[4] while ITU-T P.862 (PESQ) [5] corresponding for the purpose of speech quality assessment. Then the research on objective audio quality assessment is based on the classic model of PEAQ through exploiting better psychoacoustic models or improving the cognitive algorithms to increase the accuracy of the evaluation. But the PEAQ has certain limitations, making it unsuitable for many applications. Hence more perfect and practical methods were proposed.According to the evaluation principle, the evaluation methods can be separated into two categories, intrusive and non-intrusive. The present evaluation approach mainly stays on the intrusive evaluation by measuring the "distortion" between the reference signal and degraded signal. Since intrusive methods need mapping the distortion values to the predicted quality metric, it requests both signals are strictly synchronized. However, in some cases, the reference signal may not be available (e.g. wireless communications, voice over IP network, communication systems for real-time monitoring, etc.), the objective quality assessment must be operated without the reference signal. This led to the development of non-intrusive objective audio quality assessment. Presently, no non-intrusive audio quality assessment algorithm has yet been standardized by the ITU althou...

show abstract

“…Reviewing the results in Table 4.4 indicates that our model is competing with the model introduced in [62]. In the following, we provide a more detailed comparison of these competitive results for each iteration of the cross-validation and this will be statistically significance tested at the end of the section.…”

Section: Experiments With Quality Assessmentmentioning

confidence: 99%

Machine Learning for Non-Intrusive Speech Quality Assessment

Hakami¹

0

View full text Add to dashboard Cite

This thesis presents two studies on non-intrusive speech quality assessment methods. The first applies supervised learning methods to speech quality assessment, which is a common approach in machine learning based quality assessment. To outperform existing methods, we concentrate on enhancing the feature set. In the second study, we analyse quality assessment from a different point of view inspired by the biological brain and present the first unsupervised learning based non-intrusive quality assessment that removes the need for labelled training data. Supervised learning based, non-intrusive quality predictors generally involve the development of a regressor that maps signal features to a representation of perceived quality. The performance of the predictor largely depends on 1) how sensitive the features are to the different types of distortion, and 2) how well the model learns the relation between the features and the quality score. We improve the performance of the quality estimation by enhancing the feature set and using a contemporary machine learning model that fits this objective. We propose an augmented feature set that includes raw features that are presumably redundant. The speech quality assessment system benefits from this redundancy as it results in reducing the impact of unwanted noise in the input. Feature set augmentation generally leads to the inclusion of features that have non-smooth distributions. We introduce a new pre-processing method and re-distribute the features to facilitate the training. The evaluation of the system on the ITU-T Supplement23 database illustrates that the proposed system outperforms the popular standards and contemporary methods in the literature. The unsupervised learning quality assessment approach presented in this thesis is based on a model that is learnt from clean speech signals. Consequently, it does not need to learn the statistics of any corruption that exists in the degraded speech signals and is trained only with unlabelled clean speech samples. The quality has a new definition, which is based on the divergence between 1) the distribution of the spectrograms of test signals, and 2) the pre-existing model that represents the distribution of the spectrograms of good quality speech. The distribution of the spectrogram of the speech is complex, and hence comparing them is not trivial. To tackle this problem, we propose to map the spectrograms of speech signals to a simple latent space. Generative models that map simple latent distributions into complex distributions are excellent platforms for our work. Generative models that are trained on the spectrograms of clean speech signals learned to map the latent variable $Z$ from a simple distribution $P_Z$ into a spectrogram $X$ from the distribution of good quality speech. Consequently, an inference model is developed by inverting the pre-trained generator, which maps spectrograms of the signal under the test, $X_t$, into its relevant latent variable, $Z_t$, in the latent space. We postulate the divergence between the distribution of the latent variable and the prior distribution $P_Z$ is a good measure of the quality of speech. Generative adversarial nets (GAN) are an effective training method and work well in this application. The proposed system is a novel application for a GAN. The experimental results with the TIMIT and NOIZEUS databases show that the proposed measure correlates positively with the objective quality scores.

show abstract

“…These systems may differ in the considered features, the predicting function, or both. The work de-scribed in [60] makes use of a classifier to predict the discrete value of quality score while the works described in [23,24,25,26,27,28,29,31,61,62] apply regression methods (with shallow architectures) to estimate the subjective Mean Opinion Score (MOS) assigned to a speech file. On the other hand, approaches in [16,30] use a combination of classification and regression algorithms as the predicting function.…”

Section: Relevant Workmentioning

confidence: 99%

“…The labelled data that are publicly available for development and training of supervised learning based quality assessment is limited, and collecting more training data is usually expensive. The ITU-T coded speech data set, Supplement 23 [34], is the most well-known public labelled database that is commonly used for training or the evaluation of objective speech quality systems [14,23,24,26,28,29,44,8,46,61,62,64,66]. Supplement 23 database contains speech affected by noise, packet loss and various codecs and their corresponding subjective quality score.…”

Section: Relevant Workmentioning

confidence: 99%

“…To demonstrate the effectiveness of our system, we evaluated it on the ITU-T Supplement 23 database, which is widely used as benchmark for comparing the performance of non-intrusive systems [14,23,26,28,61,62]. The experimental results showed that our method outperforms contemporary single-ended quality assessment systems and current popular standards.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning for Non-Intrusive Speech Quality Assessment

Hakami¹

0

View full text Add to dashboard Cite

This thesis presents two studies on non-intrusive speech quality assessment methods. The first applies supervised learning methods to speech quality assessment, which is a common approach in machine learning based quality assessment. To outperform existing methods, we concentrate on enhancing the feature set. In the second study, we analyse quality assessment from a different point of view inspired by the biological brain and present the first unsupervised learning based non-intrusive quality assessment that removes the need for labelled training data. Supervised learning based, non-intrusive quality predictors generally involve the development of a regressor that maps signal features to a representation of perceived quality. The performance of the predictor largely depends on 1) how sensitive the features are to the different types of distortion, and 2) how well the model learns the relation between the features and the quality score. We improve the performance of the quality estimation by enhancing the feature set and using a contemporary machine learning model that fits this objective. We propose an augmented feature set that includes raw features that are presumably redundant. The speech quality assessment system benefits from this redundancy as it results in reducing the impact of unwanted noise in the input. Feature set augmentation generally leads to the inclusion of features that have non-smooth distributions. We introduce a new pre-processing method and re-distribute the features to facilitate the training. The evaluation of the system on the ITU-T Supplement23 database illustrates that the proposed system outperforms the popular standards and contemporary methods in the literature. The unsupervised learning quality assessment approach presented in this thesis is based on a model that is learnt from clean speech signals. Consequently, it does not need to learn the statistics of any corruption that exists in the degraded speech signals and is trained only with unlabelled clean speech samples. The quality has a new definition, which is based on the divergence between 1) the distribution of the spectrograms of test signals, and 2) the pre-existing model that represents the distribution of the spectrograms of good quality speech. The distribution of the spectrogram of the speech is complex, and hence comparing them is not trivial. To tackle this problem, we propose to map the spectrograms of speech signals to a simple latent space. Generative models that map simple latent distributions into complex distributions are excellent platforms for our work. Generative models that are trained on the spectrograms of clean speech signals learned to map the latent variable $Z$ from a simple distribution $P_Z$ into a spectrogram $X$ from the distribution of good quality speech. Consequently, an inference model is developed by inverting the pre-trained generator, which maps spectrograms of the signal under the test, $X_t$, into its relevant latent variable, $Z_t$, in the latent space. We postulate the divergence between the distribution of the latent variable and the prior distribution $P_Z$ is a good measure of the quality of speech. Generative adversarial nets (GAN) are an effective training method and work well in this application. The proposed system is a novel application for a GAN. The experimental results with the TIMIT and NOIZEUS databases show that the proposed measure correlates positively with the objective quality scores.

show abstract

Probabilistic non-intrusive quality assessment of speech for bounded-scale preference scores

Cited by 4 publications

References 13 publications

On objective assessment of audio quality — A review

On objective assessment of audio quality — A review

Machine Learning for Non-Intrusive Speech Quality Assessment

Machine Learning for Non-Intrusive Speech Quality Assessment

Contact Info

Product

Resources

About