A Semi-Supervised Learning Approach for Acoustic-Prosodic Personality Perception in Under-Resourced Domains

Abstract

Automatic personality analysis has gained attention in the last years as a fundamental dimension in human-to-human and human-to-machine interaction. However, it still suffers from limited number and size of speech corpora for specific domains, such as the assessment of children's personality. This paper investigates a semi-supervised training approach to tackle this scenario. We devise an experimental setup with age and language mismatch and two training sets: a small labeled training set from the Interspeech 2012 Personality Sub-challenge, containing French adult speech labeled with personality OCEAN traits, and a large unlabeled training set of Portuguese children's speech. As test set, a corpus of Portuguese children's speech labeled with OCEAN traits is used. Based on this setting, we investigate a weak supervision approach that iteratively refines an initial model trained with the labeled data-set using the unlabeled data-set. We also investigate knowledge-based features, which leverage expert knowledge in acoustic-prosodic cues and thus need no extra data. Results show that, despite the large mismatch imposed by language and age differences, it is possible to attain improvements with these techniques, pointing both to the benefits of using a weak supervision and expert-based acoustic-prosodic features across age and language.

Transfer Learning for Personality Perception via Speech Emotion Recognition

“…An SVM was used as the classifier for all five feature types. [23] used weak supervision to iteratively refine the initial model trained on labeled data using the unlabeled data based on knowledge-based features that exploit expert knowledge of acoustic-prosodic cues. [24] employed an autoencoder to extract features and used an adaptive neuro-fuzzy inference system to model the uncertainty in speech.…”

Automatic Personality Traits Perception Using Asymmetric Auto-Encoder

“…The frequency-domain linear prediction and Mel Frequency Cepstral Coefficient (MFCC) features were extracted in [13]. In [14], the acoustic and duration-related features, such as silence ratio, speech duration ratio, and speech rate features, were proposed. Moreover, Guidi et al evaluated the impact of median/mean of the fundamental frequency of speech signal, the frame-toframe jitter factor, and the glottal flow spectral slope features on personality recognition [15].…”

“…Studies were not limited only to the acoustic and nonverbal features extraction. The lexical [8], [16], knowledge-based features [14], and BFI questionnaire scores [17] were added to this variety of features as well. The effort toward achieving a wider variety of features reached 6,373 statistical feature in [18].…”

“…It can also be deduced from Table VI that previous methods were not effective in classifying all the five personality traits simultaneously. For example, for the method of[14], the UA and accuracy outcomes for Extraversion and Conscientiousness are significant. However, only Conscientiousness UA enhanced compared with previous studies.…”