Holistic Affect Recognition Using PaNDA: Paralinguistic Non-Metric Dimensional Analysis

Zhang, Yue; Weninger, Felix; Schuller, Björn; Picard, Rosalind W.

doi:10.1109/taffc.2019.2961881

Cited by 24 publications

(7 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Results show that, by using multi-corpus instead of single-corpus training, we can have an overall significant improvement in the UAR. We also found that multi-corpus MTL can improve the performance on a specific corpus (EmoDB) by using other corpora during the training, which is inline with previous studies [9], [10], [14].…”

Section: B Within-corpus Evaluationsupporting

confidence: 90%

“…They did not map different emotion categories from different corpora into the same subspace on purpose, as this would come with information loss. In a later work [14], other para-linguistic tasks in addition to emotion were also considered, including 18 different classification and regression tasks. A task relatedness matrix was introduced to more efficiently benefit from related tasks.…”

Section: Multi-task Learning In Cross-corpus Settingsmentioning

confidence: 99%

“…Moreover, some emotion dimensions like Valence are more sensitive to language [29], especially when using only the audio modality [1]. Despite the reported drop in performance in some studies, it has been shown in many others that using multiple corpora with different languages can still not only achieve reasonable performance but also be beneficial to deal with rare events that occur frequently in real-life [1], [9], [14], [29]. Thus, we can still benefit from using multiple corpora annotated for emotion even if they contain expressions from different languages.…”

Section: Multi-task Learning For Multi-lingual Sermentioning

confidence: 99%

“…This means that we can have one shared model across different tasks (or different corpora), where the target of each task (or corpus) is defined differently. MTL has been successfully used in SER, either on one corpus, by considering different classifiers for different emotion dimensions [12], [13], or by using different corpora, where each corpus set of labels is considered a different target [9], [14].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Multi-Corpus Affect Recognition with Emotion Embeddings and Self-Supervised Representations of Speech

Alisamir

Ringeval

Portet

2022

2022 10th International Conference on Affective Computing and Intelligent Interaction (ACII)

View full text Add to dashboard Cite

Speech emotion recognition systems use data-driven machine learning techniques that rely on annotated corpora. To achieve a usable performance in real-life, we need to exploit multiple different datasets since each one can shed the light on some specific expression of affect. However, different corpora use subjectively defined annotation schemes, which poses a challenge to train a model that can sense similar emotions across different corpora. Here, we propose a method that can relate similar emotions across corpora without being explicitly trained for it. Our method relies on self-supervised representations, which can provide us with highly contextualised speech representations, and multi-task learning paradigms. This allows to train on different corpora without changing their labelling schemes. The results show that by fine-tuning self-supervised representations on each corpus separately, we can significantly improve the state of the art within-corpus performance. We further demonstrate that by using multiple corpora during the training of the same model, we can improve the cross-corpus performance, and show that our emotion embeddings can effectively recognise the same emotions across different corpora.

show abstract

Section: B Within-corpus Evaluationsupporting

confidence: 90%

Section: Multi-task Learning In Cross-corpus Settingsmentioning

confidence: 99%

Section: Multi-task Learning For Multi-lingual Sermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multi-Corpus Affect Recognition with Emotion Embeddings and Self-Supervised Representations of Speech

Alisamir

Ringeval

Portet

2022

2022 10th International Conference on Affective Computing and Intelligent Interaction (ACII)

View full text Add to dashboard Cite

show abstract

“…Due to the bottleneck of manual data annotation, many existing datasets are labelled along one or a few target dimensions, which might also be attributable to the traditional supervised learning paradigm. For example, emotion datasets usually have different labeling schemes due to the diversity of emotional concepts (Zhang et al, 2020). Another challenge is handling data with missing labels that can happen for various reasons in the data collection process.…”

Section: Introductionmentioning

confidence: 99%

openXDATA: A Tool for Multi-Target Data Generation and Missing Label Completion

Weninger,

Zhang,

Picard

2020

Preprint

Self Cite

View full text Add to dashboard Cite

A common problem in machine learning is to deal with datasets with disjoint label spaces and missing labels. In this work, we introduce the openXDATA tool that completes the missing labels in partially labelled or unlabelled datasets in order to generate multi-target data with labels in the joint label space of the datasets. To this end, we designed and implemented the cross-data label completion (CDLC) algorithm that uses a multi-task shared-hidden-layer DNN to iteratively complete the sparse label matrix of the instances from the different datasets. We apply the new tool to estimate labels across four emotion datasets: one labeled with discrete emotion categories (e. g., happy, sad, angry), one labeled with continuous values along arousal and valence dimensions, one with both kinds of labels, and one unlabeled. Testing with drop-out of true labels, we show the ability to estimate both categories and continuous labels for all of the datasets, at rates that approached the ground truth values. openXDATA is available under the GNU General Public License from https://github.com/fweninger/openXDATA.

show abstract

Unlocking human-like conversations: Scoping review of automation techniques for personalized healthcare interventions using conversational agents

Martins,

Londral,

L. Nunes

et al. 2024

International Journal of Medical Informatics

View full text Add to dashboard Cite

Holistic Affect Recognition Using PaNDA: Paralinguistic Non-Metric Dimensional Analysis

Cited by 24 publications

References 68 publications

Multi-Corpus Affect Recognition with Emotion Embeddings and Self-Supervised Representations of Speech

Multi-Corpus Affect Recognition with Emotion Embeddings and Self-Supervised Representations of Speech

openXDATA: A Tool for Multi-Target Data Generation and Missing Label Completion

Unlocking human-like conversations: Scoping review of automation techniques for personalized healthcare interventions using conversational agents

Contact Info

Product

Resources

About