ConferencingSpeech 2022 Challenge: Non-intrusive Objective Speech Quality Assessment (NISQA) Challenge for Online Conferencing Applications

Yi, Gaoxiong; Xiao, Wei; Xiao, Yiming; Naderi, Babak; Möller, Sebastian; Wardah, Wafaa; Mittag, Gabriel; Cutler, Ross; Zhang, Zhuohuang; Williamson, Donald S.; Chen, Fei; Yang, Fuzheng; Shang, Shidong

doi:10.48550/arxiv.2203.16032

Cited by 3 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Non-intrusive objective speech quality assessment tools like ITU-T P.563 [14] do not require a reference, though it has low correlation to subjective quality [15]. Newer neural net-based methods such as [15,16,17,18] provide better correlations to subjective quality. NISQA [19] is an objective metric for P.804, though the correlation to subjective quality is not sufficient to use as a challenge metric.…”

Section: Related Workmentioning

confidence: 99%

ICASSP 2022 Acoustic Echo Cancellation Challenge

Cutler

Saabas

Pärnamaa

et al. 2022

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

The ICASSP 2023 Speech Signal Improvement Challenge is intended to stimulate research in the area of improving the speech signal quality in communication systems. The speech signal quality can be measured with SIG in ITU-T P.835 and is still a top issue in audio communication and conferencing systems. For example, in the ICASSP 2022 Deep Noise Suppression challenge, the improvement in the background and overall quality is impressive, but the improvement in the speech signal is statistically zero. To improve the speech signal the following speech impairment areas must be addressed: coloration, discontinuity, loudness, and reverberation. A dataset and test set were provided for the challenge, and the winners were determined using an extended crowdsourced implementation of ITU-T P.80's listening phase . The results show significant improvement was made across all measured dimensions of speech quality.

show abstract

Section: Related Workmentioning

confidence: 99%

ICASSP 2022 Acoustic Echo Cancellation Challenge

Cutler

Saabas

Pärnamaa

et al. 2022

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…For the PSTN and Tencent corpora, our model performs comparably to the next best model (1.7% better and 5.6% worse). But for the TUB corpus, our model performed 27.4% better than the next-closest competitor for that corpus and 42.9% better than the overall secondplace model [10]. This suggests that the proposed model performs exceptionally well on unseen data.…”

Section: Test Resultsmentioning

confidence: 83%

“…Additionally, we test various combinations of the BiLSTM and AttPoolFF components placed on top of both the XLS-R and MFCC feature extractors. We show that the XLS-R feature extractor does not only outperform the classic MFCC feature extractor, but it also results in competitive model performances with regards to the root mean squared error (RMSE) on data from the ConferencingSpeech 2022 challenge [10].…”

Section: Introductionmentioning

confidence: 99%

Pre-trained Speech Representations as Feature Extractors for Speech Quality Assessment in Online Conferencing Applications

Bastiaan¹,

Balabin²,

Vandenberghe³

et al. 2022

Interspeech 2022

View full text Add to dashboard Cite

Speech quality in online conferencing applications is typically assessed through human judgements in the form of the mean opinion score (MOS) metric. Since such a labor-intensive approach is not feasible for large-scale speech quality assessments in most settings, the focus has shifted towards automated MOS prediction through end-to-end training of deep neural networks (DNN). Instead of training a network from scratch, we propose to leverage the speech representations from the pre-trained wav2vec-based XLS-R model. However, the number of parameters of such a model exceeds task-specific DNNs by several orders of magnitude, which poses a challenge for resulting fine-tuning procedures on smaller datasets. Therefore, we opt to use pre-trained speech representations from XLS-R in a feature extraction rather than a fine-tuning setting, thereby significantly reducing the number of trainable model parameters. We compare our proposed XLS-R-based feature extractor to a Mel-frequency cepstral coefficient (MFCC)-based one, and experiment with various combinations of bidirectional long short term memory (Bi-LSTM) and attention pooling feedforward (AttPoolFF) networks trained on the output of the feature extractors. We demonstrate the increased performance of pre-trained XLS-R embeddings in terms a reduced root mean squared error (RMSE) on the ConferencingSpeech 2022 MOS prediction task.

show abstract

“…Automatic audio quality measurement can be applied in many different audio related tasks, including detecting fake-quality audio [1], monitoring speech quality for online conference [2], or assessing the naturalness of synthesized speech [3]. Besides, since many people use mobile phones in their daily life, and audio quality is one of the considerations of choosing mobile phones [4], measuring the quality of audio playback for mobile phones would also be one of the important tasks.…”

Section: Introductionmentioning

confidence: 99%

Using Classification Algorithms to Predict Taiwan Stock Market - A Case Study of Taiwan Index Futures

Wang¹,

Lee²

2022

Communications in Computer and Information Science

View full text Add to dashboard Cite

This paper proposes to use both audio input and subject information to predict the personalized preference of two audio segments with the same content in different qualities. A siamese network is used to compare the inputs and predict the preference. Several different structures for each side of the siamese network are investigated, and an LDNet with PANNs' CNN6 as the encoder and a multi-layer perceptron block as the decoder outperforms a baseline model using only audio input the most, where the overall accuracy grows from 77.56% to 78.04%. Experimental results also show that using all the subject information, including age, gender, and the specifications of headphones or earphones, is more effective than using only a part of them.

show abstract

ConferencingSpeech 2022 Challenge: Non-intrusive Objective Speech Quality Assessment (NISQA) Challenge for Online Conferencing Applications

Cited by 3 publications

References 20 publications

ICASSP 2022 Acoustic Echo Cancellation Challenge

ICASSP 2022 Acoustic Echo Cancellation Challenge

Pre-trained Speech Representations as Feature Extractors for Speech Quality Assessment in Online Conferencing Applications

Using Classification Algorithms to Predict Taiwan Stock Market - A Case Study of Taiwan Index Futures

Contact Info

Product

Resources

About