End-to-End Deep Learning Framework for Speech Paralinguistics Detection Based on Perception Aware Spectrum

Abstract

In this paper, we propose an end-to-end deep learning framework to detect speech paralinguistics using perception aware spectrum as input. Existing studies show that speech under cold has distinct variations of energy distribution on low frequency components compared with the speech under 'healthy' condition. This motivates us to use perception aware spectrum as the input to an end-to-end learning framework with small scale dataset. In this work, we try both Constant Q Transform (CQT) spectrum and Gammatone spectrum in different end-toend deep learning networks, where both spectrums are able to closely mimic the human speech perception and transform it into 2D images. Experimental results show the effectiveness of the proposed perception aware spectrum with end-to-end deep learning approach on Interspeech 2017 Computational Paralinguistics Cold sub-Challenge. The final fusion result of our proposed method is 8% better than that of the provided baseline in terms of UAR.

Speech analysis for health: Current state-of-the-art and the increasing impact of deep learning

“…E2E models rely purely on the statistics of the available data to learn optimal features, therefore it is possible that the available data, due to the imbalance, did not contain a sufficient variation for the cold class. An E2E approach was also presented by one of challenge participants [138]. In this system, Constant Q Transform (CQT) spectrum and Gammatone spectrum features were feed into 5 CNN layers followed by a single gated recurrent unit layer.…”

The DKU-LENOVO Systems for the INTERSPEECH 2019 Computational Paralinguistic Challenge

“…Deep convolutional neural network plays an important role in many areas, including paralinguistic speech attribute recognition in recent years [15,21]. The convolutional structure can capture the patterns on the images, and more generally, on spectrograms or other time-frequency representations.…”

“…DenseNet [24] connects every layer with other layers in a feedforward manner thus has the potential to reduce the problem of gradient vanishing. Compared to other network structures applied in previous paralinguistic challenges [15,21], the networks we adopt here are much deeper. Therefore we focus on the orca activity detection task due to the relatively large size of training data.…”

“…Previous works show that convolutional neural networks trained with the STFT spectrogram inputs achieved good performance on many paralinguistic problems [15,16,17]. The concept of deep embedding has also been widely used in many tasks such as speaker verification [18,19] and language identification [18,20].…”

Squeeze for Sneeze: Compact Neural Networks for Cold and Flu Recognition

“…For this task, we utilise the Upper Respiratory Tract Infection Corpus (URTIC) dataset as featured in the IN-TERSPEECH 2017 Computational Paralinguistics Challenge (COMPARE) [21]. A range of different approaches have already been undertaken on this data, from conventional OPENS-MILE [22] based systems [21,23], to more contemporary deep learning systems [24]. As the aim of the presented work is to explore the advantages of network pruning and optimisation, we opted to train standard multi-layer dense neural networks, the suitability of which have been demonstrated for this data [25].…”