Anti-transfer learning for task invariance in convolutional neural networks for speech processing

Guizzo, Eric; Weyde, Tillman; Tarroni, Giacomo

doi:10.1016/j.neunet.2021.05.012

Cited by 14 publications

(18 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A well-established solution to overcome the data scarcity in SER is transfer learning by weight initialization: network weights are initialized with values from a network that was pretrained with a different task, possibly on a different (usually large) dataset. Many variants of this method have been shown to improve the performance of SER models in limited-data scenarios and even when the task is rather distant from speech emotion [33]- [35]. Also, various data augmentation strategies have been successfully adopted for the same purpose, e.g.…”

Section: Introductionmentioning

confidence: 99%

Learning Speech Emotion Representations in the Quaternion Domain

Guizzo

Weyde

Scardapane

et al. 2023

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

The modeling of human emotion expression in speech signals is an important, yet challenging task. The high resource demand of speech emotion recognition models, combined with the general scarcity of emotion-labelled data are obstacles to the development and application of effective solutions in this field. In this paper, we present an approach to jointly circumvent these difficulties. Our method, named RH-emo, is a novel semisupervised architecture aimed at extracting quaternion embeddings from real-valued monoaural spectrograms, enabling the use of quaternion-valued networks for speech emotion recognition tasks. RH-emo is a hybrid real/quaternion autoencoder network that consists of a real-valued encoder in parallel to a real-valued emotion classifier and a quaternion-valued decoder. On the one hand, the classifier permits to optimization of each latent axis of the embeddings for the classification of a specific emotionrelated characteristic: valence, arousal, dominance, and overall emotion. On the other hand, quaternion reconstruction enables the latent dimension to develop intra-channel correlations that are required for an effective representation as a quaternion entity. We test our approach on speech emotion recognition tasks using four popular datasets: IEMOCAP, RAVDESS, EmoDB, and TESS, comparing the performance of three well-established real-valued CNN architectures (AlexNet, ResNet-50, VGG) and their quaternion-valued equivalent fed with the embeddings created with RH-emo. We obtain a consistent improvement in the test accuracy for all datasets, while drastically reducing the resources' demand of models. Moreover, we performed additional experiments and ablation studies that confirm the effectiveness of our approach. The RH-emo repository is available at: https://github.com/ispamm/rhemo.

show abstract

Section: Introductionmentioning

confidence: 99%

Learning Speech Emotion Representations in the Quaternion Domain

Guizzo

Weyde

Scardapane

et al. 2023

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…A well-established solution to overcome the data scarcity in SER is transfer learning by weight initialization: network weights are initialized with values from a network that was pretrained with a different task, possibly on a different (usually large) dataset. Many variants of this method have been shown to improve the performance of SER models in limited-data scenarios and even when the task is rather distant from speech emotion [30]- [32]. Also, various data augmentation strategies have been successfully adopted for the same purpose, e.g.…”

Section: Introductionmentioning

confidence: 99%

Learning Speech Emotion Representations in the Quaternion Domain

Guizzo¹,

Weyde²,

Scardapane³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The modeling of human emotion expression in speech signals is an important, yet challenging task. The high resource demand of speech emotion recognition models, combined with the the general scarcity of emotion-labelled data are obstacles to the development and application of effective solutions in this field. In this paper, we present an approach to jointly circumvent these difficulties. Our method, named RH-emo, is a novel semi-supervised architecture aimed at extracting quaternion embeddings from real-valued monoaural spectrograms, enabling the use of quaternion-valued networks for speech emotion recognition tasks. RH-emo is a hybrid real/quaternion autoencoder network that consists of a real-valued encoder in parallel to a real-valued emotion classifier and a quaternion-valued decoder. On the one hand, the classifier permits to optimize each latent axis of the embeddings for the classification of a specific emotionrelated characteristic: valence, arousal, dominance and overall emotion. On the other hand, the quaternion reconstruction enables the latent dimension to develop intra-channel correlations that are required for an effective representation as a quaternion entity. We test our approach on speech emotion recognition tasks using four popular datasets: Iemocap, Ravdess, EmoDb and Tess, comparing the performance of three well-established real-valued CNN architectures (AlexNet, ResNet-50, VGG) and their quaternion-valued equivalent fed with the embeddings created with RH-emo. We obtain a consistent improvement in the test accuracy for all datasets, while drastically reducing the resources' demand of models. Moreover, we performed additional experiments and ablation studies that confirm the effectiveness of our approach. The RH-emo repository is available at: https://github.com/ispamm/rhemo.

show abstract

“…During the heading and fowering stages, wheat has no evident changes in features other than the heading rate, which rapidly changes. Tus, better feature extraction and expression capacity of the regression model are required [26][27][28]. Convolutional neural network (CNN) has a high capacity for feature extraction and expression and has been widely used in image processing, video analysis, and other felds [29][30][31].…”

Section: Introductionmentioning

confidence: 99%

Determination of Wheat Heading Stage Using Convolutional Neural Networks on Multispectral UAV Imaging Data

Cao

Liu

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

The heading and flowering stages are crucial for wheat growth and should be used for fusarium head blight (FHB) and other plant prevention operations. Rapid and accurate monitoring of wheat growth in hilly areas is critical for determining plant protection operations and strategies. Currently, the operation time for FHB prevention and plant protection is primarily determined by manual tour inspection of plant growth, which has the disadvantages of low information gathering and subjectivity. In this study, an unmanned aerial vehicle (UAV) equipped with a multispectral camera was used to collect wheat canopy multispectral images and heading rate information during the heading and flowering stages in order to develop a method for detecting the appropriate time for preventive control of FHB. A 1D convolutional neural network + decision tree model (1D CNN + DT) was designed. All the multispectral information was input into the model for feature extraction and result regression. The regression revealed that the coefficient of determination (R2) between multispectral information in the wheat canopy and the heading rate was 0.95, and the root mean square error of prediction (RMSE) was 0.24. This result was superior to that obtained by directly inputting multispectral data into neural networks (NN) or by inputting multispectral data into NN via traditional VI calculation, support vector machines regression (SVR), or decision tree (DT). On the basis of FHB prevention and control production guidelines and field research results, a discrimination model for FHB prevention and plant protection operation time was developed. After the output values of the regression model were input into the discrimination model, a 97.50% precision was obtained. The method proposed in this study can efficiently monitor the growth status of wheat during the heading and flowering stages and provide crop growth information for determining the timing and strategy of FHB prevention and plant protection operations.

show abstract

Anti-transfer learning for task invariance in convolutional neural networks for speech processing

Cited by 14 publications

References 51 publications

Learning Speech Emotion Representations in the Quaternion Domain

Learning Speech Emotion Representations in the Quaternion Domain

Learning Speech Emotion Representations in the Quaternion Domain

Determination of Wheat Heading Stage Using Convolutional Neural Networks on Multispectral UAV Imaging Data

Contact Info

Product

Resources

About